What is iRESTful?

Before explaining what iRESTful is (or will be), let me explain what problems I have to deal with while I am working for a customer and how I am solving those issues.

This customer have 2 billion page views on its platform.  His platform was created by various junior developers that did not communicate well together.  So, the whole platform was composed of procedural PHP scripts.  There is no unit test and the whole platform is coupled together.  I even saw some PHP scripts that are executing some other PHP scripts before its execution to work properly.  So, these procedural scripts also have states, which were not documented anywhere.

On top of this nightmare, the platform is very unstable.  From time to time, some servers are going down for no reason, and since everything is coupled together, the whole platform is crashing when this happens.

I have to re-write the whole platform in order to make it stable.  However, supporting that many QPS while creating a code base easy to maintain is not an easy task.

My customer agreed to the re-write, but wants to use Redis as a normalized database system.  The application would read from Redis, and write in both MySQL and Redis.  However, the MySQL database must be denormalized.

If you used Redis in the past, you know that, to keep it fast, you need to make sure it doesn’t swap on disk all the time.  However, my customer had 5TB of data in its system.  Since it is impossible to have 5TB of ram on a server, I had to create a sharding system.

My goal is to create a system where we could save and retrieve an Entity object inside a normalized database system. 

That normalized database system would then retrieve objects by making only 1 call on the Redis server.

Since it is a normalized database system, the write requests would need to save the object in its hashmap key, but would also need to update the hashmap keys of the Entity objects that object contained, in cascade.  I also want to support transactions, so if one object failed to save, the whole transaction would fail.

I also need to write data in a denormalized MySQL database, mainly for business intelligence (BI) reasons. Therefore, I will write messages to RabbitMQ, then create a specific application in NodeJS that will receive these messages and create the proper transactions in MySQL.

I decided to separate my project in sub-projects, and then separate my sub-projects in packages.  For each package, I would separate the interfaces from the class implementations.

I would create a package for the interfaces to support the transactions in any database system.  Then, I would create a class implementation that use these interface to create my Redis implementation, and would create another package for the RabbitMQ implementation.

Then, I would simply use the right package, in my dependency injection declaration file, so that all write requests goes to both MySQL and Redis, but all read requests would go to Redis only.

I thought that maybe, other developers would need a similar system so I decided to build the whole system in an open source project.  I was happy to see that my customer was open to the open source idea!

Since I have to create generic objects and support them in a RESTful API, I decided to call the project iRESTful.  The whole project will be created in PHP 5.4, but since I want to support its usage thru API’s, I want to create development kits to make it easy to use in other languages/platforms as well (Android, iOS, Ruby on Rails, Python, etc.)

While working on the iRESTful platform, I will create blog posts about its progress.  Each time I finish a package or a sub-project, I will write a blog post about its usage here.

Keep in mind that the goal is to make it easy to use the whole project in your own project, or only specific sub-projects and/or packages.

Stay tuned!

EDIT: Here is how I decided to seperate the iRESTful in sub-projects.

Tags: programming