Command-Line Interface¶

The aws-sagemaker-remote CLI provides utilities to compliment processing, training, and other scripts.

Most inputs to these utilities are actually CSV strings that are processed left-to-right.

For most use-cases, pass the raw string. If the string includes a comma, it should be double-quoted.
For more advanced use-cases, pass a CSV string of operations
- Start with a string value
- Pass sagemaker to interpret the current string as a relative path in the default SageMaker bucket
- Pass json:key to interpret the current string as a local or s3 path, read that file, and get a JSON key from that file
- Pass processing:key to interpret the current string as a processing job ID and get a value from that job
- Pass training:key to interpret the current string as a training job ID and get a value from that job
- Pass batch:key to interpret the current string as an S3 batch job ID and get a value from that job

For example, when specifying the input to a script on the command line:

For known path, simply pass the value, --input s3://bucket/relative/path
Instead of hardcoding your default SageMaker bucket, use --input relative/path,sagemaker
For referencing a key named mykey in a JSON, use --input file.json,json:mykey
For a more complicated example, like referencing the output of a processing job:
- When running a processing job use --output-json relative/path.json to save the JobID to JSON
- Reference the job output by specifying --input relative/path.json,json:JobId,processing:ProcessingOutputConfig.Outputs.outputname.S3Output.S3Uri

aws-sagemaker-remote¶

Set of utilities for managing AWS training, processing, and more.

aws-sagemaker-remote [OPTIONS] COMMAND [ARGS]...

Options

--profile <profile>¶: AWS profile. Run aws-configure to configure a profile.

batch¶

S3 batch processing commands

aws-sagemaker-remote batch [OPTIONS] COMMAND [ARGS]...

report¶

Collate a report of completed and failed operations from one or more batch jobs

aws-sagemaker-remote batch report [OPTIONS]

Options

--job <job>¶

--output <output>¶

ecr¶

ECR commands

aws-sagemaker-remote ecr [OPTIONS] COMMAND [ARGS]...

build¶

Commands to build ECR images

aws-sagemaker-remote ecr build [OPTIONS] COMMAND [ARGS]...

all¶

Build all available docker images

aws-sagemaker-remote ecr build all [OPTIONS]

Options

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-inference:latest¶

Build the [aws-sagemaker-remote-inference:latest] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-inference:latest 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-inference:py27-gpu-tf¶

Build the [aws-sagemaker-remote-inference:py27-gpu-tf] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-inference:py27-gpu-tf 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-inference:py27-tf¶

Build the [aws-sagemaker-remote-inference:py27-tf] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-inference:py27-tf 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-processing:latest¶

Build the [aws-sagemaker-remote-processing:latest] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-processing:latest 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-processing:py27-gpu-tf¶

Build the [aws-sagemaker-remote-processing:py27-gpu-tf] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-processing:py27-gpu-tf 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-training:gpu¶

Build the [aws-sagemaker-remote-training:gpu] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-training:gpu 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

aws-sagemaker-remote-training:latest¶

Build the [aws-sagemaker-remote-training:latest] image

aws-sagemaker-remote ecr build aws-sagemaker-remote-training:latest 
    [OPTIONS]

Options

--path <path>¶

--tag <tag>¶

--account <account>¶

--cache, --no-cache¶

--pull, --no-pull¶

--push, --no-push¶

endpoint¶

Endpoint commands

aws-sagemaker-remote endpoint [OPTIONS] COMMAND [ARGS]...

create¶

Create a SageMaker endpoint

aws-sagemaker-remote endpoint create [OPTIONS]

Options

--name <name>¶: Name of endpoint

--config <config>¶: Name of endpoint config

--force, --no-force¶: Overwrite existing endpoint

delete¶

Delete a SageMaker endpoint

aws-sagemaker-remote endpoint delete [OPTIONS] NAME

Arguments

NAME¶: Required argument

describe¶

Describe a SageMaker endpoint

aws-sagemaker-remote endpoint describe [OPTIONS] NAME [FIELD]

Arguments

NAME¶: Required argument

FIELD¶: Optional argument

invoke¶

Invoke a SageMaker endpoint or a SageMaker-style model in a local directory

aws-sagemaker-remote endpoint invoke [OPTIONS]

Options

--name <name>¶

--model <model>¶

--variant <variant>¶

--input <input>¶

--input-glob <input_glob>¶: glob pattern if input is directory (e.g., **/*.wav)

--output <output>¶

--input-type <input_type>¶

--output-type <output_type>¶

--model-dir <model_dir>¶

endpoint-config¶

Endpoint config comands

aws-sagemaker-remote endpoint-config [OPTIONS] COMMAND [ARGS]...

create¶

Create an endpoint configuration

aws-sagemaker-remote endpoint-config create [OPTIONS]

Options

--name <name>¶

--model <model>¶

--instance-type <instance_type>¶: SageMaker instance type

--force, --no-force¶

delete¶

Delete endpoint configuration NAME

aws-sagemaker-remote endpoint-config delete [OPTIONS] NAME

Arguments

NAME¶: Required argument

describe¶

Describe endpoint configuration NAME

aws-sagemaker-remote endpoint-config describe [OPTIONS] NAME [FIELD]

Arguments

NAME¶: Required argument

FIELD¶: Optional argument

json¶

JSON manipulation commands

aws-sagemaker-remote json [OPTIONS] COMMAND [ARGS]...

parse¶

Parse comma-delimited path [PATH]

aws-sagemaker-remote json parse [OPTIONS] PATH

Arguments

PATH¶: Required argument

read¶

Read field [FIELD] from JSON file at [PATH]

aws-sagemaker-remote json read [OPTIONS] PATH FIELD

Arguments

PATH¶: Required argument

FIELD¶: Required argument

model¶

Model commands

aws-sagemaker-remote model [OPTIONS] COMMAND [ARGS]...

create¶

Create a SageMaker model

aws-sagemaker-remote model create [OPTIONS]

Options

--job <job>¶: Job name

--model-artifact <model_artifact>¶: Model artifact (S3 URI). Relative path assumes default bucket

--name <name>¶: Model name

--inference-image <inference_image>¶: ECR Docker URI for inference

--inference-image-path <inference_image_path>¶: Path for building image if necessary

--inference-image-accounts <inference_image_accounts>¶: Accounts for building image

--role <role>¶: SageMaker inference role name

--force, --no-force¶

--multimodel, --singlemodel¶: SingleModel or MultiModel mode

delete¶

Delete SageMaker model NAME

aws-sagemaker-remote model delete [OPTIONS] NAME

Arguments

NAME¶: Required argument

describe¶

Describe endpoint configuration NAME

aws-sagemaker-remote model describe [OPTIONS] NAME [FIELD]

Arguments

NAME¶: Required argument

FIELD¶: Optional argument

processing¶

Processing commands

aws-sagemaker-remote processing [OPTIONS] COMMAND [ARGS]...

describe¶

Describe training job NAME.

Print full JSON if FIELD is not specified.
Print only FIELD if specified (e.g., ModelArtifacts.S3ModelArtifacts or LastModifiedTime).

aws-sagemaker-remote processing describe [OPTIONS] NAME [FIELD]

Arguments

NAME¶: Required argument

FIELD¶: Optional argument

s3¶

S3 management commands

aws-sagemaker-remote s3 [OPTIONS] COMMAND [ARGS]...

concat¶

Download and concatenate multiple files from an S3 manifest

aws-sagemaker-remote s3 concat [OPTIONS]

Options

--manifest <manifest>¶: Input manifest file

--limit <limit>¶: Number of files (0 for all)

--output <output>¶: Required Output file path

upload¶

Upload a file or directory to S3, optionally with GZIP

aws-sagemaker-remote s3 upload [OPTIONS] SRC DST

Options

--gz, --no-gz¶

--root <root>¶: Target path in resulting zip file

Arguments

SRC¶: Required argument

DST¶: Required argument

training¶

SageMaker training commands

aws-sagemaker-remote training [OPTIONS] COMMAND [ARGS]...

describe¶

Describe training job NAME.

Print full JSON if FIELD is not specified.
Print only FIELD if specified (e.g., ModelArtifacts.S3ModelArtifacts or LastModifiedTime).

aws-sagemaker-remote training describe [OPTIONS] NAME [FIELD]

Arguments

NAME¶: Required argument

FIELD¶: Optional argument

transform¶

SageMaker batch transform commands

aws-sagemaker-remote transform [OPTIONS] COMMAND [ARGS]...

create¶

Create a batch transformation job for objects in S3

Model must already exist in SageMaker
Model instances are deployed
Each S3 object is posted to one of your instances
Results are saved in S3 with the extension “.out”
Model instances are destroyed

aws-sagemaker-remote transform create [OPTIONS]

Options

--base-job-name <base_job_name>¶: Transform job base name. If job name not provided, job name is the base job name plus a timestamp.

--job-name <job_name>¶: Transform job name for tracking in AWS console

--model-name <model_name>¶: Required SageMaker Model name

--concurrency <concurrency>¶: Concurrency (number of concurrent requests to each container)

--timeout <timeout>¶: Timeout in seconds per request

--retries <retries>¶: Number of retries for each failed request

--input-s3 <input_s3>¶: Required Input path on S3

--output-s3 <output_s3>¶: Required Output path on S3

--input-type <input_type>¶: Required Input MIME type (“Content-Type” header)

--output-type <output_type>¶: Required Output MIME type (“Accept” header)

--output-json <output_json>¶: Save job information in JSON file

--instance-type <instance_type>¶: SageMaker Instance type (e.g., ml.m5.large)

--instance-count <instance_count>¶: Number of containers to use (processing will be distributed)

--payload-mb <payload_mb>¶: Maximum payload size (MB)