Command-Line Interface¶
The aws-sagemaker-remote
CLI provides utilities to compliment
processing, training, and other scripts.
Most inputs to these utilities are actually CSV strings that are processed left-to-right.
- For most use-cases, pass the raw string. If the string includes a comma, it should be double-quoted.
- For more advanced use-cases, pass a CSV string of operations
- Start with a string value
- Pass
sagemaker
to interpret the current string as a relative path in the default SageMaker bucket - Pass
json:key
to interpret the current string as a local or s3 path, read that file, and get a JSON key from that file - Pass
processing:key
to interpret the current string as a processing job ID and get a value from that job - Pass
training:key
to interpret the current string as a training job ID and get a value from that job - Pass
batch:key
to interpret the current string as an S3 batch job ID and get a value from that job
For example, when specifying the input to a script on the command line:
- For known path, simply pass the value,
--input s3://bucket/relative/path
- Instead of hardcoding your default SageMaker bucket, use
--input relative/path,sagemaker
- For referencing a key named
mykey
in a JSON, use--input file.json,json:mykey
- For a more complicated example, like referencing the output of a processing job:
- When running a processing job use
--output-json relative/path.json
to save the JobID to JSON - Reference the job output by specifying
--input relative/path.json,json:JobId,processing:ProcessingOutputConfig.Outputs.outputname.S3Output.S3Uri
- When running a processing job use
aws-sagemaker-remote¶
Set of utilities for managing AWS training, processing, and more.
aws-sagemaker-remote [OPTIONS] COMMAND [ARGS]...
Options
-
--profile
<profile>
¶ AWS profile. Run aws-configure to configure a profile.
ecr¶
ECR commands
aws-sagemaker-remote ecr [OPTIONS] COMMAND [ARGS]...
build¶
Commands to build ECR images
aws-sagemaker-remote ecr build [OPTIONS] COMMAND [ARGS]...
all¶
Build all available docker images
aws-sagemaker-remote ecr build all [OPTIONS]
Options
-
--cache
,
--no-cache
¶
-
--pull
,
--no-pull
¶
-
--push
,
--no-push
¶
aws-sagemaker-remote-inference:latest¶
Build the [aws-sagemaker-remote-inference:latest] image
aws-sagemaker-remote ecr build aws-sagemaker-remote-inference:latest
[OPTIONS]
Options
-
--path
<path>
¶
-
--tag
<tag>
¶
-
--account
<account>
¶
-
--cache
,
--no-cache
¶
-
--pull
,
--no-pull
¶
-
--push
,
--no-push
¶
aws-sagemaker-remote-inference:py27-gpu-tf¶
Build the [aws-sagemaker-remote-inference:py27-gpu-tf] image
aws-sagemaker-remote ecr build aws-sagemaker-remote-inference:py27-gpu-tf
[OPTIONS]
Options
-
--path
<path>
¶
-
--tag
<tag>
¶
-
--account
<account>
¶
-
--cache
,
--no-cache
¶
-
--pull
,
--no-pull
¶
-
--push
,
--no-push
¶
aws-sagemaker-remote-inference:py27-tf¶
Build the [aws-sagemaker-remote-inference:py27-tf] image
aws-sagemaker-remote ecr build aws-sagemaker-remote-inference:py27-tf
[OPTIONS]
Options
-
--path
<path>
¶
-
--tag
<tag>
¶
-
--account
<account>
¶
-
--cache
,
--no-cache
¶
-
--pull
,
--no-pull
¶
-
--push
,
--no-push
¶
aws-sagemaker-remote-processing:latest¶
Build the [aws-sagemaker-remote-processing:latest] image
aws-sagemaker-remote ecr build aws-sagemaker-remote-processing:latest
[OPTIONS]
Options
-
--path
<path>
¶
-
--tag
<tag>
¶
-
--account
<account>
¶
-
--cache
,
--no-cache
¶
-
--pull
,
--no-pull
¶
-
--push
,
--no-push
¶
aws-sagemaker-remote-processing:py27-gpu-tf¶
Build the [aws-sagemaker-remote-processing:py27-gpu-tf] image
aws-sagemaker-remote ecr build aws-sagemaker-remote-processing:py27-gpu-tf
[OPTIONS]
Options
-
--path
<path>
¶
-
--tag
<tag>
¶
-
--account
<account>
¶
-
--cache
,
--no-cache
¶
-
--pull
,
--no-pull
¶
-
--push
,
--no-push
¶
endpoint¶
Endpoint commands
aws-sagemaker-remote endpoint [OPTIONS] COMMAND [ARGS]...
create¶
Create a SageMaker endpoint
aws-sagemaker-remote endpoint create [OPTIONS]
Options
-
--name
<name>
¶ Name of endpoint
-
--config
<config>
¶ Name of endpoint config
-
--force
,
--no-force
¶
Overwrite existing endpoint
delete¶
Delete a SageMaker endpoint
aws-sagemaker-remote endpoint delete [OPTIONS] NAME
Arguments
-
NAME
¶
Required argument
describe¶
Describe a SageMaker endpoint
aws-sagemaker-remote endpoint describe [OPTIONS] NAME [FIELD]
Arguments
-
NAME
¶
Required argument
-
FIELD
¶
Optional argument
invoke¶
Invoke a SageMaker endpoint or a SageMaker-style model in a local directory
aws-sagemaker-remote endpoint invoke [OPTIONS]
Options
-
--name
<name>
¶
-
--model
<model>
¶
-
--variant
<variant>
¶
-
--input
<input>
¶
-
--input-glob
<input_glob>
¶ glob pattern if input is directory (e.g., **/*.wav)
-
--output
<output>
¶
-
--input-type
<input_type>
¶
-
--output-type
<output_type>
¶
-
--model-dir
<model_dir>
¶
endpoint-config¶
Endpoint config comands
aws-sagemaker-remote endpoint-config [OPTIONS] COMMAND [ARGS]...
create¶
Create an endpoint configuration
aws-sagemaker-remote endpoint-config create [OPTIONS]
Options
-
--name
<name>
¶
-
--model
<model>
¶
-
--instance-type
<instance_type>
¶ SageMaker instance type
-
--force
,
--no-force
¶
json¶
JSON manipulation commands
aws-sagemaker-remote json [OPTIONS] COMMAND [ARGS]...
model¶
Model commands
aws-sagemaker-remote model [OPTIONS] COMMAND [ARGS]...
create¶
Create a SageMaker model
aws-sagemaker-remote model create [OPTIONS]
Options
-
--job
<job>
¶ Job name
-
--model-artifact
<model_artifact>
¶ Model artifact (S3 URI). Relative path assumes default bucket
-
--name
<name>
¶ Model name
-
--inference-image
<inference_image>
¶ ECR Docker URI for inference
-
--inference-image-path
<inference_image_path>
¶ Path for building image if necessary
-
--inference-image-accounts
<inference_image_accounts>
¶ Accounts for building image
-
--role
<role>
¶ SageMaker inference role name
-
--force
,
--no-force
¶
-
--multimodel
,
--singlemodel
¶
SingleModel or MultiModel mode
s3¶
S3 management commands
aws-sagemaker-remote s3 [OPTIONS] COMMAND [ARGS]...
transform¶
SageMaker batch transform commands
aws-sagemaker-remote transform [OPTIONS] COMMAND [ARGS]...
create¶
Create a batch transformation job for objects in S3
- Model must already exist in SageMaker
- Model instances are deployed
- Each S3 object is posted to one of your instances
- Results are saved in S3 with the extension “.out”
- Model instances are destroyed
aws-sagemaker-remote transform create [OPTIONS]
Options
-
--base-job-name
<base_job_name>
¶ Transform job base name. If job name not provided, job name is the base job name plus a timestamp.
-
--job-name
<job_name>
¶ Transform job name for tracking in AWS console
-
--model-name
<model_name>
¶ Required SageMaker Model name
-
--concurrency
<concurrency>
¶ Concurrency (number of concurrent requests to each container)
-
--timeout
<timeout>
¶ Timeout in seconds per request
-
--retries
<retries>
¶ Number of retries for each failed request
-
--input-s3
<input_s3>
¶ Required Input path on S3
-
--output-s3
<output_s3>
¶ Required Output path on S3
-
--input-type
<input_type>
¶ Required Input MIME type (“Content-Type” header)
-
--output-type
<output_type>
¶ Required Output MIME type (“Accept” header)
-
--output-json
<output_json>
¶ Save job information in JSON file
-
--instance-type
<instance_type>
¶ SageMaker Instance type (e.g., ml.m5.large)
-
--instance-count
<instance_count>
¶ Number of containers to use (processing will be distributed)
-
--payload-mb
<payload_mb>
¶ Maximum payload size (MB)