Browsed by
Tag: #athena

Querying CloudTrail Logs from a centralized S3 Location using Athena

Querying CloudTrail Logs from a centralized S3 Location using Athena

Comments 0 Comment

Hello, it’s been some time since my last blog post. Today, I want to discuss the process of querying AWS CloudTrail logs stored in a centralized S3 bucket using AWS Athena. But before diving into the topic, let’s briefly introduce AWS CloudTrail and AWS Athena. AWS CloudTrail is a service that provides governance, compliance, and auditing capabilities for your AWS account. It records all user actions performed on AWS services and resources as events. By aggregating logs from multiple accounts,…

Read More Read More

Covid-19 Analysis with BigData Applications – Part 3

Covid-19 Analysis with BigData Applications – Part 3

Comments 0 Comment

Hi there! If you’ve been following these blog series, we are looking at a BigData project to analyze Covid-19 data. So far, we have looked at the overall architecture and ETL Spark jobs. In this post, let’s look the scheduler component (Lambda function) in this workflow. The main reason I’m using Lambda here is due to its serverless nature and native integration with other AWS services. For example, we could trigger it via a CloudWatch scheduled rule in regular basis…

Read More Read More