Custom NoSQL - storing data in files and not only in a database. Storing settings, small data and caching files

03.07.21 IT / Database 5882

Almost any site or web application processes various data, such data can be of very different sizes. It is generally accepted to store data in databases (DB) and use the SQL language to perform various operations on them. But you can resort to using other storage methods, for example, store data in files .

This approach appeared quite a long time ago, one can even say it was used initially, because in order to save any setting of a program or application, it was enough to create a file in the same folder and save the value in it. Later, database management systems (DBMS) and relational databases began to appear, which were designed to streamline the work with data processing, to make operations standardized and secure. However, the disadvantage was the slowdown in performance and the need for database design.

Gradually, non-relational databases began to appear, where data storage is implemented not only in tables, but also in ordinary files of various convenient formats. Today, there is a lot of such software (software), and it has been tacitly described by a concept called NoSQL (not only SQL) . In addition to its advantages, which include ease of use and speed , the NoSQL approach has its drawbacks. For example, the need to control the work with the database using your own code, otherwise conflicts and data loss are possible.

Often, when developing your project, you may need to save minor or temporary data, settings and cache. In this case, it is not always advisable to use a classic database with SQL or a database with NoSQL. Another approach may be suitable for this - to store data in files, and some data in the existing database in a serialized or compressed form.

Storing settings

A simple example, storage of settings or basic CMS configuration - a regular file with any extension is suitable for this. Using the means of a programming language (for example, PHP), you can directly create a directory for such a file and the file itself from the code, then write the settings there and, if necessary, open the file and read the data for later use in the code.

Storing temporary or small data

If you need to save small data, you can also use file storage. However, it is necessary to consider whether it will be necessary to further complicate the structure and increase the volume of such data. If so, in this case, the best option would be to save the data immediately in a prepared database table with the possibility of further modernizing the structure.

Storing data in the database in serialized form

Small pieces of data, including temporary data, can be stored in the database in a serialized form. For example, PHP has a convenient serialize function for this and an inverse unserialize - after serialization, the data can be saved to the database and, if necessary, unloaded, deserialized and used in the code.

To store temporary or small data, you can create a special table in the database, which will consist of three columns:

- id (identifier) ​​;

- name (data name) ;

- data .

For convenient work, you can write an API to work with this table. It can contain basic operations:

- set (serialize then add or update) ;

- get ;

- delete .

The convenience lies in the fact that you do not need to specially create new tables for non-essential data.

Storing files with cache

Cache files can also be useful - they can contain complex data , otherwise the server would need to perform unnecessary operations every time. Such files can be created in a special directory of the system and cleaned up automatically when the entire cache or one section is cleared (for example, sections can be named by the names of system modules).