Megafon: Speeding up services with Tarantool

A telecom operator of MegaFon’s level manages an incredible workload.

MegaFon already has 80 million users. Its billing system processes 2 billion transactions.
200 TB of data change slightly slower. All this data is live, not historical; it’s used in a single billing process.
5,000 servers in 14 data centers ensure service operation and payment.

In 2016, the external audit of the IT landscape in MegaFon has shown that it is necessary to improve fault tolerance and availability of basic structures: billing, ERP, operations and business support systems. Almost all of them use Oracle as the single data storage. This means that at peak loads, the ascending structure may fail, which negatively affected the subscriber service.

MegaFon has decided to move to a microservice architecture and rebuild the IT landscape for a three-layer model. This move solved the load on the underlying systems and accelerated time to market.

The concept assumes that basic structures are located on the bottom layer, which practically do not change. This includes the core of billing, communication systems, ERP.
In the middle are microservices, which are updated every few weeks. This layer contains caches, data marts, marketing systems, etc.

The top layer is the most mobile. The client interfaces are presented here, so it takes several days from setting a business task to updating.

It has been decided to take off the load from the core to the middle-level microservices using a lightweight and reliable storage. MegaFon has formed the following requirements:

in-memory solution
milliseconds for writes and reads
horizontal scaling
100,000 write and read operations per second
TCO within the agreed budget

Moreover, the vendor should be ready to evolve the product.

The modern architecture of MegaFon allows creating microservices based on caches, lifting data from different domains. The operator stores data in the billing system, while Tarantool storefronts help to quickly interact with them. In just two years of cooperation, the MegaFon and Tarantool teams have jointly created more than 25 services.

One of the first projects was the user’s product data mart. The data from the storefront is used in the personal and mobile account. The service stores information about all relevant products of the user, calculates and makes personalized commercial offers. This data mart uses data from 6 servers in 3 data centers, processes 700 gigabytes of data in RAM, and creates 3,000 records per second. Updates from the master system arrive at the data mart with a one-second lag.

Tarantool is used as a high-performance solution in the MegaFon real-time marketing (RTM) project. The project is designed to personalize the approach to each client and respond to events in real time. Several Tarantool storefronts solve the issue of fast access to data. RTM marts retrieve and process about 1.5TB of data.

Another popular service is the tariff calculator. The service shows the interface to the client, caches price parameters from Oracle, and calculates an offer according to the business logic right inside Tarantool. In fact, the calculator gives the required cost of tariffs virtually without delay, in real time.

Tarantool does not replace Oracle but complements it
80mln

Tarantool-based solutions support services for 80,000,000 subscribers in 10 time zones
25+

25+ services within MegaFon use Tarantool with regard to geographic redundancy and receive updates without downtime
turnkey

Tarantool experts provide turnkey services and help create an internal competence center in MegaFon
80+

The developers of MegaFon create Tarantool-based solutions on their own: the internal development team has grown from 4 people to 80+ in two years