Ebizu App Publisher objective is to provide an app analytics, engagement tool that focuses on highly accurate location intelligence, deep insights on users and monetizing mobile audience data. Ebizu own and operate the largest location and beacon network in Southeast Asia that can be utilize to trigger actions or collect data.
The project itself is hosted on AWS. Highest traffic is ~50 Million hit endpoint / Day. And we receive average of 5 GB / Day in
S3 with CSV format.
Database Design & System Architecture
The database will be use PostgreSQL and hosted on AWS RDS. Why Postgre? Because we need
JSONfeature, and postgre has
JSONB. At first
JSONBwas great, but we are not following "Use the right tool for the right things". So we move Cold Data to
RDSused for Warm Data.
Read:Write ratio is 2:8. Ebizu App Publisher will capture every data, and send it into Realtime Stream and Batch Stream.
Unlimited Background Jobs
To make the API still response fast, every process is moved into background jobs, fortunately
Lumenhas this out of the box. We are heavily using
AWS SQS, and
Redisfor long delay Job.
Auto Scaling Servers
We have this issue with background jobs server, or we called
cliservers. We use
SQSfor auto scaling trigger, based on how many on current queue.
Build the fastest API
With default settings of
elastic beanstalkwe can achieve ~50 Million hit endpoint per day, but we still can improve it. We have plan to use Swoole.
Because we have our data in
S3we can utilize
Athenato query our data.
SNSto the rescue.
PostGISto the rescue, but we found another awesome tool to replace this job, tile38.
We can make an event when user enter geofence, for example, send push notification, send sms, send phone call, etc.
2016 Jan - 2017 Aug
Architecting Big Data Application on top of AWS is awesome.