Addressing Complexity and Security in Distributed Systems Development
Updated: Oct 29
Usman Wajid, James Tryend (ICE)
Overcomplexity is a feature of many large and distributed IT systems. This is something that often results from building distributed systems over time, using components from multiple teams/vendors, sometimes even developed in different technologies. With the ever-increasing risks related to cybersecurity and privacy threats, the complexity of distributed systems can be a hindrance to maintaining and ensuring privacy and security controls at the component and overall system level.
In a way the complexity of the system can help maintain security if it is a consequence of the techniques used to constrain cybersecurity and privacy threats. The flip side is that complexity can also hinder security if it reduces the ability to audit, coordinate and respond to proper use and threats.
In the last decade, the models and techniques specifically focusing on the management (including secrurity) and maintenance of distributed IT systems have received significant attention. System development techniques (such as agile and dev-ops) combined with deployment models (such as virtualisation and containers) have contributed towards significantly reducing the complexity of large and distributed IT systems. With the industry-wide adoption of common approaches for systems development and resource management, the ability to manage and maintain complex distributed systems becomes much more straightforward.
The adoption of common standards for systems development, management and security ensures certain level of transparency, which is pivotal to reducing complexity and increasing trust both by developers and by end end-users of IT systems. The agreement on common standards particularly for distributed systems development and management can bring similar benefits to the IT domain that the organisations in the logistics domain are enjoying through the adoption of standard for shipping containers.
In essence, the adoption of common standards for the management and security of complex systems provides two main benefits. The first is the reduction in the number of interoperability and integration issues. Secondly, standardisation brings together a broader audience to address impeding issues. It creates a large community around specific problems, where only a small number of contributors are needed to help benefit the larger community. It is the similar process that has allowed open-source software to gain such an impressive reputation for being secure and being able to efficiently find and fix issues.
Being able to deploy and manage distributed components of the system in a homogenous approach gives another benefit in the form of having discrete segmentation for the different elements. Through the adotion of standards, it becomes a natural fit to secure and segment the pieces in isolation and then be able to explicitly coordinate the integration aspects of those components. This also allows to develop auditing mechanisms that measure the performance of not only individual components but also system as a whole e.g. with regards to cybersecurity and privacy threats. Thus, by making the use of standards explicit, the management and maintenance of distribued system becomes more consistent. Other expected impacts include improved trustworthiness of the IT system, acceleration of the development and implementation of auditing and certification processes, better assessment of compliance procedures and improved market opportunities for vendors of security and privacy solutions.
At the end, the federated platform development activities in the EFPF project adopt best practices of distributed systems development, including but not limited to:
The use of container based development and deployment approach for all IT components
The use of open and widely used standards for security and privacy controls at the user authorisation, access control and data exchange levels
The use of common (de-facto) standards for communication and data exchange at the shop-floor, sysems and business/enterprise levels
The agreements on the use of common systems and appraoches for continuous development and integration activities
To stay up to date with the EFPF activities stay tuned with the project website at: www.efpf.org