Backend

Perhaps the most challenging part of the system

The backend has three tasks in the system:

- receive data
- store received data
- share stored data

Roughly speaking, all backend technology choices and specifications belong to one of the above areas. The choices should also support several different areas: API interfaces; possible file transfers using different mechanisms and in different formats; reporting; batch runs, etc. In addition to these, the data must be validated, protected, and its referential integrity must be taken care of, etc.

In addition to these, the information must be found/obtained quickly using various search criteria and updates must be seen in real time.

Due to the above, it might be a good idea to first create a concept analysis that defines the key concepts of the business and the relationships between them (e.g. What is a customer and what information is related to them. What products and features are there. What is the customer/product life cycle, etc.)

After this, it is possible to start defining what/how and with what tools things will be done. In this context, it is good to take into account existing solutions (e.g. user management) and their boundary conditions.

How?

As in application development in general, there is no single right way to do this. Concept analysis can be done on paper, by creating database tables or with a database design tool (e.g. DBeaver, DataGrip). What is essential is that the plan can be made and the tool used to make it is easy enough and does not take time to learn. Once the plan is ready, it must be modeled as a working data structure in the selected storage system (SQL database, NoSQL database, BigData, ObjectStore, disk system, etc.). A working data structure can also consist of several storage mechanisms (e.g. a reference to a video in a relational database and the actual video in an objectstore or bigdisk system).

The actual creation of tables and relations between them in the database and the management of changes to tables can be something that requires attention and consideration of processes for how these situations are managed. Many programming languages ​​have implemented a so-called ORM model for processing database data, and this model also makes it possible to create all tables and relations between them. Another model is to create the tables and database manually and process the database using SQL/NoSQL. ORM may be easier for the coder, while SQL/NoSQL-based implementations may be able to utilize database features more effectively and the database structure may become more efficient.

In real life, the final model is often somewhere between ORM and SQL/NoSQL.

Once the data model and data storage have been implemented, it is time to build interfaces for data processing. These include REST/SOAP/"something else" APIs, integrations, possibly built data warehouses for reporting, etc. In all data receiving and sending, user permissions must be taken into account and the received data must be validated. Clear error reporting must also be planned at this stage.

All of this makes the implementation of the backend perhaps the most challenging part of the system.