AWS Databases and Analytics

Redshift

1. Use Case

2. Large-scale SQL analytical database

3. Querying in Redshift is FAST

4. Redshift isn’t a complete replacement for a Hadoop system (no streaming, no text processing)

5. Get data into Redshift:

6. COPY from S3 (delimited text files)

7. COPY from DynamoDB (NoSQL datastore)
8. JDBC/ODBC transactions (not efficient for bulk loading)

Tables have ‘keys’ that define how the data is split across slices. The recommended practice is to split based upon commonly-joined columns, so that joined data resides on the same slice, thus avoiding the need to move data between systems.

1. Examples:

COPY table1 FROM 's3://bucket1/' credentials 'aws_access_key_id=abc;aws_secret_access_key=xyz' delimiter '|' gzip removequotes truncatecolumns maxerror 1000
SELECT DISTINCT field1 FROM table1
SELECT COUNT(DISTINCT field2) FROM table1

EMR

• EMR FAQs
• Extract, Transform, and Load (ETL) Data with Amazon EMR