Theory of constraints

Although it is related to management and perhaps production or manufacturing systems, but since in any software code, some parts of the code do the management and have some control over the other operational parts or resources we can use this theory to study software programs and software ecosystems too.

If you have enough of resources like memory, cpu, disk and ...., and your load is low and don't expect more performance perhaps this "theory of constraints" (or TOC for short) can be just a theory for you. But if you are dealing with a lack of resources or low-performance TOC might help you to think about your software design again and perhaps modify some parts to get better performance or better use of available resources.



What TOC in a nutshell says is that in any process (small or big, simple or complex) there is, at least, one constraint which limits the process to get better performance from the available resources. And if you try and be successful to remove this constraint another one will appear. So there is always, at least, one constrains which is your process bottleneck to get better performance.

Let me give you an example. The picture shows a processing diagram of a software code. You can also consider it as some individual software components working together as an ecosystem or integrated system like what we already talked about it. The diagram shows everything, no need to detail description.

This is one of the typical processing models we usually use to process stored data on disk. We read them, distribute them on similar workers then collect the results and store them.

Suppose our goal is getting the best possible performance, and note that we know that P1 has no data producing limit, so P1 can produce millions of data packet per seconds for example. Now the question is what are the constraints?

The first thing appears in my mind is read or write speed of the disk. In fact, if you can't get the performance you like, you can easily change the disk to one of those solid-state drives to see if performance gets better or not.

What else? Again, since we usually think we are all high skilled programmers, we say now the problem is CPU or Memory. OK, we get a leading edge, super fast, with enough fast memory, and ... with SSD and run the application on it. This computer is the best we can do for the hardware. Now after we all hardware parts get changed and we make sure that we can do nothing else with hardware, it is time to look at the software itself.

We usually have one reading and writing thread. We need to look at these threads code to make them more efficient. P2 reads data packets and distributes them to P3 processes. The problem usually is these small processes, in fact, we have used 3 thread for P3 processes for some reason, the reason is each of P3s need more time to process a data packet than P2 needs to write or P4 needs to read from. If you increase the number of workers (I mean P3) at some point the performance goes down, because of locking mechanisms P2 or P4 needs to use to distribute or collect data packets or the wait/idle times P2 or P4 needs to have because P3s are either overload or have no data to deliver.

Now you see you can't improve performance at the time you reach to a point that increasing the P3 works lower the performance. Yes, there is always constraints, you may change the programming language, you may change the design, you may even design the software to be able to run on the cloud at that moment you still have many constraints.

But it is most important to identify them, prioritize them and then try to overcome them one by one. My experience shows that most of the time the major bottlenecks are from our design and programming, not the hardware, programming language, ... , or OS and ... Although when you get through all programming bottleneck aspects it is time to overcome the networking, hardware and ...

At the bottom line, there are always many constraints in our mind as programmers or software designers too, we can overcome them by having other experts points of view. Then we have time, money, other tasks, the life problem, emotions ... and at this point, you say, "sorry I can't do anything more". This is why all known operating systems still have some bugs and performance issues they had years ago.