/ 
Net-Amazon-EMR-0.10
River stage zero No dependents
/ Net::Amazon::EMR

NAME

Net::Amazon::EMR - API for Amazon's Elastic Map-Reduce service

SYNOPSIS

use Net::Amazon::EMR;

my $emr = Net::Amazon::EMR->new(
  AWSAccessKeyId  => $AWS_ACCESS_KEY_ID,
  SecretAccessKey => $SECRET_ACCESS_KEY,
  ssl             => 1,
  );

my $id = $emr->run_job_flow(Name => "Example Job",
                            Instances => {
                                Ec2KeyName => 'myKeyId',
                                InstanceCount => 10,
                                KeepJobFlowAliveWhenNoSteps => 1,
                                MasterInstanceType => 'm1.small',
                                Placement => { AvailabilityZone => 'us-east-1a' },
                                SlaveInstanceType => 'm1.small',
                            },
  );

print "Job flow id = " . $id->JobFlowId . "\n";

# Get details of just-launched job
$result = $emr->describe_job_flows(JobFlowIds => [ $id->JobFlowId ]);

# or get details of all jobs created after a given time
$result = $emr->describe_job_flows(CreatedAfter => '2012-12-17T07:19:57Z');

# or use DateTime
$result = $emr->describe_job_flows(CreatedAfter => DateTime->new(year => 2012, month => 12, day => 17));

# See the details of the typed result
use Data::Dumper; print Dumper($result);

# or dispense with types and see the details as a perl hash
use Data::Dumper; print Dumper($result->as_hash);

# Flexible Booleans - 1, 0, undef, 'true', 'false'
$emr->set_visible_to_all_users(JobFlowIds => $id, VisibleToAllUsers => 1);
$emr->set_termination_protection(JobFlowIds => [ $id->JobFlowId ], TerminationProtected => 'false');

DESCRIPTION

This is an implementation of the Amazon Elastic Map-Reduce API.

CONSTRUCTOR

new(%options)

This is the constructor. Options are as follows:

  • AWSAccessKeyId (required)

    Your AWS access key.

  • SecretAccessKey (required)

    Your secret key.

  • ssl (optional)

    If set to a true value, the base_url will use https:// instead of http://. Defaults to true.

METHODS

Detailed information on each of the methods can be found in the Amazon EMR API documentation. Each method takes a hash of parameters using the names given in the documentation. Parameter passing uses the following rules:

  • Array inputs such as InstanceGroups.member.N use their primary name and a Perl ArrayRef, i.e. InstanceGroups => [ ... ] in this example.

  • Either hashes or object instances may be passed in; e.g both of the following forms are acceptable:

    $emr->run_job_flow(Name => "API Test Job",
                            Instances => {
                                Ec2KeyName => 'xxx',
                                InstanceCount => 1,
                            },
    );

$emr->run_job_flow(Name => "API Test Job",
                            Instances => Net::Amazon::EMR::JobFlowInstancesConfig->new(
                                Ec2KeyName => 'xxx',
                                InstanceCount => 1,
                            ),
    );

  • Otherwise, the names of parameters are exactly as found in the Amazon documentation for API version 2009-03-31.

add_instance_groups(%params)

AddInstanceGroups adds an instance group to a running cluster. Returns a Net::Amazon::EMR::AddInstanceGroupsResult object.

add_job_flow_steps(%params)

AddJobFlowSteps adds new steps to a running job flow. Returns 1 on success.

describe_job_flows(%params)

Returns a Net::Amazon::EMR::RunJobFlowResult that describes the job flows that match all of the supplied parameters.

modify_instance_groups(%params)

Modifies the number of nodes and configuration settings of an instance group. Returns 1 on success.

run_job_flow(%params)

Creates and starts running a new job flow. Returns a Net::Amazon::EMR::RunJobFlowResult object that contains the job flow ID.

set_termination_protection(%params)

Locks a job flow so the Amazon EC2 instances in the cluster cannot be terminated by user intervention, an API call, or in the event of a job-flow error. Returns 1 on success.

set_visible_to_all_users(%params)

Sets whether all AWS Identity and Access Management (IAM) users under your account can access the specifed job flows. Returns 1 on success.

terminate_job_flows(%params)

Terminates a list of job flows. Returns 1 on success.

ERROR HANDLING

If an error occurs in any of the methods, the error will be logged and an Exception::Class exception of type Net::Amazon::EMR::Exception will be thrown.

ERROR LOGGING

Logging uses Log::Log4perl. You should initialise Log::Log4perl at the beginning of your program to suit your needs. The simplest way to enable debugging output to STDERR is to call

use Log::Log4perl qw/:easy/;
Log::Log4perl->easy_init($DEBUG);

AUTHOR

Jon Schutz

http://notes.jschutz.net

BUGS

Please report any bugs or feature requests to bug-net-amazon-emr at rt.cpan.org, or through the web interface at http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Net-Amazon-EMR. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes.

SUPPORT

You can find documentation for this module with the perldoc command.

perldoc Net::Amazon::EMR

You can also look for information at:

ACKNOWLEDGEMENTS

The core interface code was adapted from Net::Amazon::EC2.

LICENSE AND COPYRIGHT

Copyright 2012 Jon Schutz.

This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

See http://dev.perl.org/licenses/ for more information.

SEE ALSO

Amazon EMR API: http://http://docs.amazonwebservices.com/ElasticMapReduce/latest/APIReference/Welcome.html