Note
REST API v1.1 corresponds to Sahara v0.3.X and Sahara Icehouse release
REST API v1.1 enhances the Sahara REST API v1.0 and includes all requests from v1.0. REST API V1.1 is Elastic Data Processing (EDP) REST API. It covers the majority of new functions related to creating job binaries and job objects on running Hadoop clusters.
Description
A Data Source object provides the location of input or output for MapReduce jobs and may reference different types of storage. Sahara doesn’t perform any validation checks for data source locations.
Data Source ops
| Verb | URI | Description |
|---|---|---|
| GET | /v1.1/{tenant_id}/data-sources | Lists all Data Sources |
| GET | /v1.1/{tenant_id}/data-sources/<data_source_id> | Shows information about specified Data Source by id |
| POST | /v1.1/{tenant_id}/data-sources | Create a new Data Source |
| DELETE | /v1.1/{tenant_id}/data-sources/<data_source_id> | Removes specified Data Source |
Examples
Normal Response Code: 200 (OK)
Errors: none
This operation returns the list of all created data sources.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/data-sources
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"data_sources": [
{
"description": "This is input",
"url": "swift://container.sahara/text",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-09 12:37:19.295701",
"updated_at": null,
"type": "swift",
"id": "151d0c0c-464f-4724-96a6-4732d0ca62e1",
"name": "input"
},
{
"description": "This is output",
"url": "swift://container.sahara/result",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-09 12:37:58.155911",
"updated_at": null,
"type": "swift",
"id": "577e8bd8-b105-46f0-ace7-baee61e0adda",
"name": "output"
},
{
"description": "This is hdfs input",
"url": "hdfs://test-master-node:8020/user/hadoop/input",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2014-01-23 12:37:24.720387",
"updated_at": null,
"type": "hdfs",
"id": "63e3d1e6-52d0-4d27-ab8a-f8e236ded200",
"name": "hdfs_input"
}
]
}
Normal Response Code: 200 (OK)
Errors: none
This operation shows information about a specified Data Source.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/data-sources/151d0c0c-464f-4724-96a6-4732d0ca62e1
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"data_source": {
"description": "",
"url": "swift://container.sahara/text",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-09 12:37:19.295701",
"updated_at": null,
"type": "swift",
"id": "151d0c0c-464f-4724-96a6-4732d0ca62e1",
"name": "input"
}
}
Normal Response Code: 202 (ACCEPTED)
Errors: none
This operation returns the created Data Source.
request
POST http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/data-sources
{
"description": "This is input",
"url": "swift://container.sahara/text",
"credentials": {
"password": "swordfish",
"user": "admin"
},
"type": "swift",
"name": "text"
}
response
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"data_source": {
"description": "This is input",
"url": "swift://container.sahara/text",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 11:15:25.971886",
"type": "swift",
"id": "af7dc864-6331-4c30-80f5-63d74b667eaf",
"name": "text"
}
}
Example:
This example creates an hdfs data source.
request
POST http://sahara:8386/v1.1/e262c255a7de4a0ab0434bafd75660cd/data-sources{ "description": "This is hdfs input", "url": "hdfs://test-master-node:8020/user/hadoop/input", "type": "hdfs", "name": "hdfs_input" }response
HTTP/1.1 202 ACCEPTED Content-Type: application/json{ "data_source": { "description": "This is hdfs input", "url": "hdfs://test-master-node:8020/user/hadoop/input", "tenant_id": "e262c255a7de4a0ab0434bafd75660cd", "created_at": "2014-01-23 12:37:24.720387", "type": "hdfs", "id": "63e3d1e6-52d0-4d27-ab8a-f8e236ded200", "name": "hdfs_input" } }
Normal Response Code: 204 (NO CONTENT)
Errors: none
Removes Data Source
This operation returns nothing.
This operation does not require a request body.
request
DELETE http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/data-sources/af7dc864-6331-4c30-80f5-63d74b667eaf
response
HTTP/1.1 204 NO CONTENT
Content-Type: application/json
Description
Job Binary Internals are objects for storing job binaries in the Sahara internal database. A Job Binary Internal contains raw data of executable Jar files, Pig or Hive scripts.
Job Binary Internal ops
| Verb | URI | Description |
|---|---|---|
| GET | /v1.1/{tenant_id}/job-binary-internals | Lists all Job Binary Internals |
| GET | /v1.1/{tenant_id}/job-binary-internals/<job_binary_internal_id> | Shows info about specified Job Binary Internal by id |
| PUT | /v1.1/{tenant_id}/job-binary-internals/<name> | Create a new Job Binary Internal with specified name |
| DELETE | /v1.1/{tenant_id}/job-binary-internals/<job_binary_internal_id> | Removes specified Job Binary Internal |
| GET | /v1.1/{tenant_id}/job-binary-internals/<job_binary_internal_id>/data | Retrieves data of specified Job Binary Internal |
Examples
Normal Response Code: 200 (OK)
Errors: none
This operation returns the list of all stored Job Binary Internals.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binary-internals
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"binaries": [
{
"name": "example.pig",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:36:59.329034",
"updated_at": null,
"datasize": 161,
"id": "d2498cbf-4589-484a-a814-81436c18beb3"
},
{
"name": "udf.jar",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:43:52.008620",
"updated_at": null,
"datasize": 3745,
"id": "22f1d87a-23c8-483e-a0dd-cb4a16dde5f9"
}
]
}
Normal Response Code: 200 (OK)
Errors: none
This operation shows information about a specified Job Binary Internal.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binary-internals/d2498cbf-4589-484a-a814-81436c18beb3
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"job_binary_internal": {
"name": "example.pig",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:36:59.329034",
"updated_at": null,
"datasize": 161,
"id": "d2498cbf-4589-484a-a814-81436c18beb3"
}
}
Normal Response Code: 202 (ACCEPTED)
Errors: none
This operation shows information about the uploaded Job Binary Internal.
The request body should contain raw data (file) or script text.
request
PUT http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binary-internals/script.pig
response
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"job_binary_internal": {
"name": "script.pig",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 13:17:35.994466",
"updated_at": null,
"datasize": 160,
"id": "4833dc4b-8682-4d5b-8a9f-2036b47a0996"
}
}
Normal Response Code: 204 (NO CONTENT)
Errors: none
Removes Job Binary Internal object from Sahara’s db
This operation returns nothing.
This operation does not require a request body.
request
DELETE http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binary-internals/4833dc4b-8682-4d5b-8a9f-2036b47a0996
response
HTTP/1.1 204 NO CONTENT
Content-Type: application/json
Normal Response Code: 200 (OK)
Errors: none
Retrieves data of specified Job Binary Internal object.
This operation returns raw data.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binary-internals/4248975-3c82-4206-a58d-6e7fb3a563fd/data
response
HTTP/1.1 200 OK
Content-Length: 161
Content-Type: text/html; charset=utf-8
Description
Job Binaries objects are designed to create links to certain binaries stored either in the Sahara internal database or in Swift.
Job Binaries ops
| Verb | URI | Description |
|---|---|---|
| GET | /v1.1/{tenant_id}/job-binaries | Lists all Job Binaries |
| GET | /v1.1/{tenant_id}/job-binaries/<job_binary_id> | Shows info about specified Job Binary by id |
| POST | /v1.1/{tenant_id}/job-binaries | Create a new Job Binary object |
| DELETE | /v1.1/{tenant_id}/job-binaries/<job_binary_id> | Removes specified Job Binary |
| GET | /v1.1/{tenant_id}/job-binaries/<job_binary_id>/data | Retrieves data of specified Job Binary |
Examples
Normal Response Code: 200 (OK)
Errors: none
This operation returns the list of all created Job Binaries.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binaries
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"binaries": [
{
"description": "",
"url": "internal-db://d2498cbf-4589-484a-a814-81436c18beb3",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:36:59.375060",
"updated_at": null,
"id": "84248975-3c82-4206-a58d-6e7fb3a563fd",
"name": "example.pig"
},
{
"description": "",
"url": "internal-db://22f1d87a-23c8-483e-a0dd-cb4a16dde5f9",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:43:52.265899",
"updated_at": null,
"id": "508fc62d-1d58-4412-b603-bdab307bb926",
"name": "udf.jar"
},
{
"description": "",
"url": "swift://container/jar-example.jar",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 14:25:04.970513",
"updated_at": null,
"id": "a716a9cd-9add-4b12-b1b6-cdb71aaef350",
"name": "jar-example.jar"
}
]
}
Normal Response Code: 200 (OK)
Errors: none
This operation shows information about a specified Job Binary.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binaries/a716a9cd-9add-4b12-b1b6-cdb71aaef350
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"job_binary": {
"description": "",
"url": "swift://container/jar-example.jar",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 14:25:04.970513",
"updated_at": null,
"id": "a716a9cd-9add-4b12-b1b6-cdb71aaef350",
"name": "jar-example.jar"
}
}
Normal Response Code: 202 (ACCEPTED)
Errors: none
This operation shows information about the created Job Binary.
request
POST http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binaries
{
"url": "swift://container/jar-example.jar",
"name": "jar-example.jar",
"description": "This is job binary",
"extra": {
"password": "swordfish",
"user": "admin"
}
}
response
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"job_binary": {
"description": "This is job binary",
"url": "swift://container/jar-example.jar",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 14:49:20.106452",
"id": "07f86352-ee8a-4b08-b737-d705ded5ff9c",
"name": "jar-example.jar"
}
}
Normal Response Code: 204 (NO CONTENT)
Errors: none
Removes Job Binary object
This operation returns nothing.
This operation does not require a request body.
request
DELETE http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binaries/07f86352-ee8a-4b08-b737-d705ded5ff9c
response
HTTP/1.1 204 NO CONTENT
Content-Type: application/json
Normal Response Code: 200 (OK)
Errors: none
Retrieves data of specified Job Binary object.
This operation returns raw data.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/job-binaries/84248975-3c82-4206-a58d-6e7fb3a563fd/data
response
HTTP/1.1 200 OK
Content-Length: 161
Content-Type: text/html; charset=utf-8
Description
Job objects represent Hadoop jobs. A Job object contains lists of all binaries needed for job execution. User should provide data sources and Job parameters to start job execution. A Job may be run on an existing cluster or a new transient cluster may be created for the Job run.
Job ops
| Verb | URI | Description |
|---|---|---|
| GET | /v1.1/{tenant_id}/jobs | Lists all created Jobs |
| GET | /v1.1/{tenant_id}/jobs/<job_id> | Shows info about specified Job by id |
| POST | /v1.1/{tenant_id}/jobs | Create a new Job object |
| DELETE | /v1.1/{tenant_id}/jobs/<job_id> | Removes specified Job |
| GET | /v1.1/{tenant_id}/jobs/config-hints/<job_type> | Shows default configuration by specified Job type |
| POST | /v1.1/{tenant_id}/jobs/<job_id>/execute | Starts Job executing |
Examples
Normal Response Code: 200 (OK)
Errors: none
This operation returns the list of all created Jobs.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/jobs
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"jobs": [
{
"description": "",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-16 11:26:54.109123",
"mains": [
{
"description": "",
"url": "internal-db://d2498cbf-4589-484a-a814-81436c18beb3",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:36:59.375060",
"updated_at": null,
"id": "84248975-3c82-4206-a58d-6e7fb3a563fd",
"name": "example.pig"
}
],
"updated_at": null,
"libs": [
{
"description": "",
"url": "internal-db://22f1d87a-23c8-483e-a0dd-cb4a16dde5f9",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:43:52.265899",
"updated_at": null,
"id": "508fc62d-1d58-4412-b603-bdab307bb926",
"name": "udf.jar"
}
],
"type": "Pig",
"id": "65afed9c-dad7-4658-9554-b7b4e1ca908f",
"name": "pig-job"
},
{
"description": "",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-16 11:29:55.008351",
"mains": [],
"updated_at": null,
"libs": [
{
"description": "This is job binary",
"url": "swift://container/jar-example.jar",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 16:03:37.979630",
"updated_at": null,
"id": "8955b12f-ed32-4152-be39-5b7398c3d04c",
"name": "hadoopexamples.jar"
}
],
"type": "Jar",
"id": "7600373c-d262-45c6-845f-77f339f3e503",
"name": "jar-job"
}
]
}
Normal Response Code: 200 (OK)
Errors: none
This operation returns the information about the specified Job.
This operation does not require a request body.
request
GET http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/jobs/7600373c-d262-45c6-845f-77f339f3e503
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"job": {
"description": "",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-16 11:29:55.008351",
"mains": [],
"updated_at": null,
"libs": [
{
"description": "This is job binary",
"url": "swift://container/jar-example.jar",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 16:03:37.979630",
"updated_at": null,
"id": "8955b12f-ed32-4152-be39-5b7398c3d04c",
"name": "hadoopexamples.jar"
}
],
"type": "Jar",
"id": "7600373c-d262-45c6-845f-77f339f3e503",
"name": "jar-job"
}
}
Normal Response Code: 202 (ACCEPTED)
Errors: none
This operation shows information about the created Job object.
request
POST http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/jobs
{
"description": "This is pig job example",
"mains": ["84248975-3c82-4206-a58d-6e7fb3a563fd"],
"libs": ["508fc62d-1d58-4412-b603-bdab307bb926"],
"type": "Pig",
"name": "pig-job-example"
}
response
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"job": {
"description": "This is pig job example",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-17 09:52:20.957275",
"mains": [
{
"description": "",
"url": "internal-db://d2498cbf-4589-484a-a814-81436c18beb3",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:36:59.375060",
"updated_at": null,
"id": "84248975-3c82-4206-a58d-6e7fb3a563fd",
"name": "example.pig"
}
],
"libs": [
{
"description": "",
"url": "internal-db://22f1d87a-23c8-483e-a0dd-cb4a16dde5f9",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-15 12:43:52.265899",
"updated_at": null,
"id": "508fc62d-1d58-4412-b603-bdab307bb926",
"name": "udf.jar"
}
],
"type": "Pig",
"id": "3cb27eaa-2f88-4c75-ab81-a36e2ab58d4e",
"name": "pig-job-example"
}
}
Normal Response Code: 204 (NO CONTENT)
Errors: none
Removes the Job object
This operation returns nothing.
This operation does not require a request body.
request
DELETE http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/jobs/07f86352-ee8a-4b08-b737-d705ded5ff9c
response
HTTP/1.1 204 NO CONTENT
Content-Type: application/json
Normal Response Code: 200 (OK)
Errors: none
This operation returns hints for configuration parameters which can be applied during job execution.
This operation does not require a request body.
Note This REST call is used just for hints and doesn’t force the user to apply any of them.
request
GET http://sahara/v1.1/11587919cc534bcbb1027a161c82cf58/jobs/config-hints/MapReduce
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"job_config": {
"configs": [
{
"name": "mapred.reducer.new-api",
"value": "true",
"description": ""
},
{
"name": "mapred.mapper.new-api",
"value": "true",
"description": ""
},
{
"name": "mapred.input.dir",
"value": "",
"description": ""
},
{
"name": "mapred.output.dir",
"value": "",
"description": ""
},
{
"name": "mapred.mapoutput.key.class",
"value": "",
"description": ""
},
{
"name": "mapred.mapoutput.value.class",
"value": "",
"description": ""
},
{
"name": "mapred.output.key.class",
"value": "",
"description": ""
},
{
"name": "mapred.output.value.class",
"value": "",
"description": ""
},
{
"name": "mapreduce.map.class",
"value": "",
"description": ""
},
{
"name": "mapreduce.reduce.class",
"value": "",
"description": ""
},
{
"name": "mapred.mapper.class",
"value": "",
"description": ""
},
{
"name": "mapred.reducer.class",
"value": "",
"description": ""
}
],
"args": []
}
}
Normal Response Code: 202 (ACCEPTED)
Errors: none
This operation returns the created Job Execution object. Note that different job types support different combinations of configs, args, and params. The Elastic Data Processing (EDP) document discusses these differences.
request
POST http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/jobs/65afed9c-dad7-4658-9554-b7b4e1ca908f/execute
{
"cluster_id": "776e441b-5816-4d47-9e07-7ded58f9a5f6",
"input_id": "af7dc864-6331-4c30-80f5-63d74b667eaf",
"output_id": "b63780f3-13d7-4286-b731-88270fb204de",
"job_configs": {
"configs": {
"mapred.map.tasks": "1",
"mapred.reduce.tasks": "1"
},
"args": ["arg1", "arg2"],
"params": {
"param2": "value2",
"param1": "value1"
}
}
}
response
HTTP/1.1 202 ACCEPTED
Content-Type: application/json
{
"job_execution": {
"output_id": "b63780f3-13d7-4286-b731-88270fb204de",
"info": {
"status": "Pending"
},
"job_id": "65afed9c-dad7-4658-9554-b7b4e1ca908f",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"created_at": "2013-10-17 13:17:03.631362",
"input_id": "af7dc864-6331-4c30-80f5-63d74b667eaf",
"cluster_id": "776e441b-5816-4d47-9e07-7ded58f9a5f6",
"job_configs": {
"configs": {
"mapred.map.tasks": "1",
"mapred.reduce.tasks": "1"
},
"args": ["arg1", "arg2"],
"params": {
"param2": "value2",
"param1": "value1"
}
},
"id": "fb2ba667-1162-4f6d-ba77-662c04dfac35"
}
}
Example execution of a Java job:
The main class is specified with edp.java.main_class. The input/output paths are passed in args because Java jobs do not use data sources. Finally, the swift configs must be specified because the input/output paths are swift paths.
request
POST http://sahara:8386/v1.1/11587919cc534bcbb1027a161c82cf58/jobs/65afed9c-dad7-4658-9554-b7b4e1ca908f/execute{ "cluster_id": "776e441b-5816-4d47-9e07-7ded58f9a5f6", "job_configs": { "configs": { "fs.swift.service.sahara.username": "myname", "fs.swift.service.sahara.password": "mypassword", "edp.java.main_class": "org.apache.hadoop.examples.WordCount" }, "args": ["swift://integration.sahara/demo/make_job.sh", "swift://integration.sahara/friday"] } }response
HTTP/1.1 202 ACCEPTED Content-Type: application/json{ "job_execution": { "output_id": null, "info": { "status": "Pending" }, "job_id": "8236b1b4-e1b8-46ef-9174-355cd4234b62", "tenant_id": "a4e4599e87e04bf1996862ae295f6f53", "created_at": "2014-02-05 23:31:57.752897", "input_id": null, "cluster_id": "466a2b6d-df00-4310-b985-c106f5231ec0", "job_configs": { "configs": { "edp.java.main_class": "org.apache.hadoop.examples.WordCount", "fs.swift.service.sahara.password": "myname", "fs.swift.service.sahara.username": "mypassword" }, "args": [ "swift://integration.sahara/demo/make_job.sh", "swift://integration.sahara/friday" ] }, "id": "724709bf-2268-46ed-8daf-47898b4630b4" } }
Description
A Job Execution object represents a Hadoop Job executing on specified cluster. A Job Execution polls the status of a running Job and reports it to the user. Also a user has the ability to cancel a running job.
Job Executions ops
| Verb | URI | Description |
|---|---|---|
| GET | /v1.1/{tenant_id}/job-executions | Lists all Job Executions |
| GET | /v1.1/{tenant_id}/job-executions/<job_execution_id> | Shows info about specified Job Execution by id |
| GET | /v1.1/{tenant_id}/job-executions/<job_execution_id>/refresh-status | Refreshes status and shows info about specified Job by id |
| GET | /v1.1/{tenant_id}/job-executions/<job_execution_id>/cancel | Cancels specified Job by id |
| DELETE | /v1.1/{tenant_id}/job-executions/<job_execution_id> | Removes specified Job |
Examples
Normal Response Code: 200 (OK)
Errors: none
This operation returns the list of all Job Executions.
This operation does not require a request body.
request
GET http://sahara/v1.1/11587919cc534bcbb1027a161c82cf58/job-executions
response
HTTP/1.1 200 OK
Content-Type: application/json
{
"job_executions": [
{
"output_id": "b63780f3-13d7-4286-b731-88270fb204de",
"info": {
"status": "RUNNING",
"externalId": null,
"run": 0,
"startTime": "Thu, 17 Oct 2013 13:53:14 GMT",
"appName": "job-wf",
"lastModTime": "Thu, 17 Oct 2013 13:53:17 GMT",
"actions": [
{
"status": "OK",
"retries": 0,
"transition": "job-node",
"stats": null,
"startTime": "Thu, 17 Oct 2013 13:53:14 GMT",
"cred": "null",
"errorMessage": null,
"externalId": "-",
"errorCode": null,
"consoleUrl": "-",
"toString": "Action name[:start:] status[OK]",
"externalStatus": "OK",
"conf": "",
"type": ":START:",
"trackerUri": "-",
"externalChildIDs": null,
"endTime": "Thu, 17 Oct 2013 13:53:15 GMT",
"data": null,
"id": "0000000-131017135256789-oozie-hado-W@:start:",
"name": ":start:"
},
{
"status": "RUNNING",
"retries": 0,
"transition": null,
"stats": null,
"startTime": "Thu, 17 Oct 2013 13:53:15 GMT",
"cred": "null",
"errorMessage": null,
"externalId": "job_201310171352_0001",
"errorCode": null,
"consoleUrl": "http://edp-master-001:50030/jobdetails.jsp?jobid=job_201310171352_0001",
"toString": "Action name[job-node] status[RUNNING]",
"externalStatus": "RUNNING",
"conf": "<pig xmlns=\"uri:oozie:workflow:0.2\">\r\n <job-tracker>edp-master-001:8021</job-tracker>\r\n <name-node>hdfs://edp-master-001:8020</name-node>\r\n <configuration>\r\n <property>\r\n <name>fs.swift.service.sahara.password</name>\r\n <value>swordfish</value>\r\n </property>\r\n <property>\r\n <name>fs.swift.service.sahara.username</name>\r\n <value>admin</value>\r\n </property>\r\n </configuration>\r\n <script>example.pig</script>\r\n <param>INPUT=swift://container.sahara/text</param>\r\n <param>OUTPUT=swift://container.sahara/output</param>\r\n</pig>",
"type": "pig",
"trackerUri": "edp-master-001:8021",
"externalChildIDs": null,
"endTime": null,
"data": null,
"id": "0000000-131017135256789-oozie-hado-W@job-node",
"name": "job-node"
}
],
"acl": null,
"consoleUrl": "http://edp-master-001.novalocal:11000/oozie?job=0000000-131017135256789-oozie-hado-W",
"appPath": "hdfs://edp-master-001:8020/user/hadoop/pig-job/9ceb6469-4d06-474d-995d-76fbc3b8c617/workflow.xml",
"toString": "Workflow id[0000000-131017135256789-oozie-hado-W] status[RUNNING]",
"user": "hadoop",
"conf": "<configuration>\r\n <property>\r\n <name>user.name</name>\r\n <value>hadoop</value>\r\n </property>\r\n <property>\r\n <name>oozie.use.system.libpath</name>\r\n <value>true</value>\r\n </property>\r\n <property>\r\n <name>nameNode</name>\r\n <value>hdfs://edp-master-001:8020</value>\r\n </property>\r\n <property>\r\n <name>jobTracker</name>\r\n <value>edp-master-001:8021</value>\r\n </property>\r\n <property>\r\n <name>oozie.wf.application.path</name>\r\n <value>hdfs://edp-master-001:8020/user/hadoop/pig-job/9ceb6469-4d06-474d-995d-76fbc3b8c617/workflow.xml</value>\r\n </property>\r\n</configuration>",
"parentId": null,
"createdTime": "Thu, 17 Oct 2013 13:53:14 GMT",
"group": null,
"endTime": null,
"id": "0000000-131017135256789-oozie-hado-W"
},
"job_id": "65afed9c-dad7-4658-9554-b7b4e1ca908f",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"start_time": "2013-10-17T17:53:14",
"updated_at": "2013-10-17 13:53:32.227919",
"return_code": null,
"oozie_job_id": "0000000-131017135256789-oozie-hado-W",
"input_id": "af7dc864-6331-4c30-80f5-63d74b667eaf",
"end_time": null,
"cluster_id": "eb85e8a0-510c-489f-b78e-ad1d29e957c8",
"id": "e63bdc21-0126-4fd2-90c6-5163d16f31df",
"progress": null,
"job_configs": {},
"created_at": "2013-10-17 13:51:11.671977"
},
{
"output_id": "b63780f3-13d7-4286-b731-88270fb204de",
"info": {
"status": "Pending"
},
"job_id": "65afed9c-dad7-4658-9554-b7b4e1ca908f",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"start_time": null,
"updated_at": null,
"return_code": null,
"oozie_job_id": null,
"input_id": "af7dc864-6331-4c30-80f5-63d74b667eaf",
"end_time": null,
"cluster_id": "eb85e8a0-510c-489f-b78e-ad1d29e957c8",
"id": "e63bdc21-0126-4fd2-90c6-5163d16f31df",
"progress": null,
"job_configs": {},
"created_at": "2013-10-17 14:37:04.107096"
}
]
}
Normal Response Code: 200 (OK)
Errors: none
This operation shows the information about a specified Job Execution.
This operation does not require a request body.
request
GET http://sahara/v1.1/11587919cc534bcbb1027a161c82cf58/job-executions/e63bdc21-0126-4fd2-90c6-5163d16f31df
response
HTTP/1.1 200 OK
Content-Type: application/json
Response body contains Job Execution object
Normal Response Code: 200 (OK)
Errors: none
This operation refreshes the status of the specified Job Execution and shows its information.
This operation does not require a request body.
request
GET http://sahara/v1.1/11587919cc534bcbb1027a161c82cf58/job-executions/4a911624-1e25-4650-bd1d-382d19695708/refresh-status
response
HTTP/1.1 200 OK
Content-Type: application/json
Response body contains Job Execution object
Normal Response Code: 200 (OK)
Errors: none
This operation cancels specified Job Execution.
This operation does not require a request body.
request
GET http://sahara/v1.1/11587919cc534bcbb1027a161c82cf58/job-executions/4a911624-1e25-4650-bd1d-382d19695708/refresh-status
response
HTTP/1.1 200 OK
Content-Type: application/json
Response body contains Job Execution object with Job Execution in KILLED state
Normal Response Code: 204 (NO CONTENT)
Errors: none
Remove an existing Job Execution.
This operation returns nothing.
This operation does not require a request body.
request
DELETE http://sahara/v1.1/job-executions/<job-execution-id>/d7g51a-8123-424e-sdsr3-eb222ec989b1
response
HTTP/1.1 204 NO CONTENT
Content-Type: application/json
The following json response represents a Job Execution object returned from Sahara
{
"output_id": "b63780f3-13d7-4286-b731-88270fb204de",
"info": {
"status": "RUNNING",
"externalId": null,
"run": 0,
"startTime": "Thu, 17 Oct 2013 13:53:14 GMT",
"appName": "job-wf",
"lastModTime": "Thu, 17 Oct 2013 13:53:17 GMT",
"actions": [
{
"status": "OK",
"retries": 0,
"transition": "job-node",
"stats": null,
"startTime": "Thu, 17 Oct 2013 13:53:14 GMT",
"cred": "null",
"errorMessage": null,
"externalId": "-",
"errorCode": null,
"consoleUrl": "-",
"toString": "Action name[:start:] status[OK]",
"externalStatus": "OK",
"conf": "",
"type": ":START:",
"trackerUri": "-",
"externalChildIDs": null,
"endTime": "Thu, 17 Oct 2013 13:53:15 GMT",
"data": null,
"id": "0000000-131017135256789-oozie-hado-W@:start:",
"name": ":start:"
},
{
"status": "RUNNING",
"retries": 0,
"transition": null,
"stats": null,
"startTime": "Thu, 17 Oct 2013 13:53:15 GMT",
"cred": "null",
"errorMessage": null,
"externalId": "job_201310171352_0001",
"errorCode": null,
"consoleUrl": "http://edp-master-001:50030/jobdetails.jsp?jobid=job_201310171352_0001",
"toString": "Action name[job-node] status[RUNNING]",
"externalStatus": "RUNNING",
"conf": "<pig xmlns=\"uri:oozie:workflow:0.2\">\r\n <job-tracker>edp-master-001:8021</job-tracker>\r\n <name-node>hdfs://edp-master-001:8020</name-node>\r\n <configuration>\r\n <property>\r\n <name>fs.swift.service.sahara.password</name>\r\n <value>swordfish</value>\r\n </property>\r\n <property>\r\n <name>fs.swift.service.sahara.username</name>\r\n <value>admin</value>\r\n </property>\r\n </configuration>\r\n <script>example.pig</script>\r\n <param>INPUT=swift://container.sahara/text</param>\r\n <param>OUTPUT=swift://container.sahara/output</param>\r\n</pig>",
"type": "pig",
"trackerUri": "edp-master-001:8021",
"externalChildIDs": null,
"endTime": null,
"data": null,
"id": "0000000-131017135256789-oozie-hado-W@job-node",
"name": "job-node"
}
],
"acl": null,
"consoleUrl": "http://edp-master-001.novalocal:11000/oozie?job=0000000-131017135256789-oozie-hado-W",
"appPath": "hdfs://edp-master-001:8020/user/hadoop/pig-job/9ceb6469-4d06-474d-995d-76fbc3b8c617/workflow.xml",
"toString": "Workflow id[0000000-131017135256789-oozie-hado-W] status[RUNNING]",
"user": "hadoop",
"conf": "<configuration>\r\n <property>\r\n <name>user.name</name>\r\n <value>hadoop</value>\r\n </property>\r\n <property>\r\n <name>oozie.use.system.libpath</name>\r\n <value>true</value>\r\n </property>\r\n <property>\r\n <name>nameNode</name>\r\n <value>hdfs://edp-master-001:8020</value>\r\n </property>\r\n <property>\r\n <name>jobTracker</name>\r\n <value>edp-master-001:8021</value>\r\n </property>\r\n <property>\r\n <name>oozie.wf.application.path</name>\r\n <value>hdfs://edp-master-001:8020/user/hadoop/pig-job/9ceb6469-4d06-474d-995d-76fbc3b8c617/workflow.xml</value>\r\n </property>\r\n</configuration>",
"parentId": null,
"createdTime": "Thu, 17 Oct 2013 13:53:14 GMT",
"group": null,
"endTime": null,
"id": "0000000-131017135256789-oozie-hado-W"
},
"job_id": "65afed9c-dad7-4658-9554-b7b4e1ca908f",
"tenant_id": "11587919cc534bcbb1027a161c82cf58",
"start_time": "2013-10-17T17:53:14",
"updated_at": "2013-10-17 13:53:32.227919",
"return_code": null,
"oozie_job_id": "0000000-131017135256789-oozie-hado-W",
"input_id": "af7dc864-6331-4c30-80f5-63d74b667eaf",
"end_time": null,
"cluster_id": "eb85e8a0-510c-489f-b78e-ad1d29e957c8",
"id": "e63bdc21-0126-4fd2-90c6-5163d16f31df",
"progress": null,
"job_configs": {},
"created_at": "2013-10-17 13:51:11.671977"
}