Enabling local load balancing, by building a highly scalable OCPP proxy

Can you give us an overview of the project and what inspired its inception?

For this project, the client faced a specific challenge in integrating a set of local devices at various charging locations with their existing Charge Point Management System (CPMS). At each charging site, such as a parking garage, multiple chargers were connected to a single local controller hardware running a Linux distribution with custom software. This software performed local load balancing operations on the connected chargers, and received sensor values from the electrical panel in a feedback loop. However, this software could not communicate upstream using OCPP, the industry-standard protocol for charger-to-backend messaging. We needed to devise a solution to address this issue, as an alternative would have been a costly integration via custom APIs and protocols directly modifying the code base of the CPMS. In summary, the objective was to enable these local controllers to communicate with the CPMS using the standard OCPP protocol, an increasingly urgent need due to several planned new deployments.

What specific problem did the client approach you with, and how did you define the project’s scope based on that problem?

The local controllers communicated externally using three different technologies: REST API, GraphQL, and a Service Bus. Our solution design involved creating an OCPP proxy server to emulate the individual chargers via the OCPP protocol. From the CPMS perspective, it controlled these chargers as if they were regular stand-alone charge points. However, in practice, a single system managed all connections and mapped communication downstream to the distributed local controllers using the three mentioned protocols. These controllers then forwarded commands to the individual chargers at that respective location. This bi-directional scheme utilized each protocol for its specific purpose. For example, session meter values were monitored on the bus, while configuring the controller or handling general charging events was managed via GraphQL or REST API.

Given the tight timeline, we adopted a Minimum Viable Product (MVP) approach, starting with the simplest solution and iterating to more advanced features as needed. An early design decision was to support only OCPP version 1.5. Although this version was older, its HTTP-based client-server model was straightforward to implement, and the CPMS had reliably operated with this version for years, covering all essential use cases for the near future. To address scalability and performance challenges—given that the system would manage a vast number of controllers and chargers—we opted for a serverless approach using Node.js, hosted in the Amazon AWS cloud.

What were the primary technical challenges presented by the client’s requirements, and how did your choice of technology stack address these challenges? Can you describe the technology stack used in the project and the reasons behind these choices?

Client’s Requirements and Challenges:

As mentioned, the client’s requirements centered on building a system capable of managing bidirectional communication via standard OCPP between charging stations connected to a custom hardware and software solution, and the already existing Charge Point Management System (CPMS). The system needed to be highly available, scale in response to network demand, and ensure data consistency across different message formats and protocols. Additionally, maintaining and monitoring system health through regular heartbeat checks was vital, as was the ability to efficiently log and access historical performance.

Technology Stack:

Our technology stack was carefully chosen to address these multifaceted requirements

Node.js & TypeScript:

We selected Node.js for its event-driven, non-blocking I/O model, which excels at handling concurrent connections, making it well-suited for the real-time communication needs of our system. TypeScript was employed to add a layer of type safety, enhancing code quality and maintainability, crucial for our complex message transformation operations.

AWS Lambda:

For our heartbeat scheduler and nightly log organization tasks, AWS Lambda was the optimal choice. It allowed us to execute code in response to events without provisioning or managing servers. This service matched our demand for reliable execution of scheduled tasks and automated scaling.

AWS Fargate:

Deploying our solution on AWS Fargate addressed the client’s scalability and fault tolerance concerns. Fargate’s serverless infrastructure offered automatic scaling and high availability without the need to manage the underlying compute instances.

Amazon DynamoDB:

We employed DynamoDB to store mappings of device and charge station IDs, as well as all transaction IDs. DynamoDB’s fast and predictable performance and seamless scalability made it perfect for ensuring quick access to our data mappings and maintaining consistency across systems.

Caching Mechanism:

Implementing a caching layer for ID mappings reduced latency and improved performance by avoiding repetitive queries to DynamoDB. This approach provided a responsive experience and cost savings by minimizing database read operations.

Amazon CloudWatch & S3:

Logs were initially written to Amazon CloudWatch for real-time monitoring and alerting. To ensure long-term storage, cost-effectiveness, and ease of access, we used scheduled AWS Lambda functions to transfer logs from CloudWatch to organized S3 buckets nightly. This practice facilitated efficient log management, enabling us to retain detailed records necessary for debugging, compliance, and analysis purposes.

What were some of the biggest technical challenges you faced during the development, and how did you overcome them?

One of the most significant technical challenges we encountered during the development of the solution was ensuring its scalability and reliability. As our system serves as a central communication hub, handling a multitude of incoming requests from various controllers and transforming them into OCPP 1.5 messages for the CPMS, it was imperative that the system could scale to meet demand without interruption or performance degradation.

Scalability Challenge:

Initially, managing the scaling of our infrastructure to accommodate fluctuating loads was complex. The system had to support a growing number of charge points and handle an increasing volume of messages while maintaining low latency and high reliability.

Solution with AWS Fargate:

To address this, we opted to deploy our Proxy system on AWS Fargate, a serverless compute engine for containers. Fargate abstracted away the underlying server and cluster management, allowing us to focus on the application development and deployment. This serverless approach meant that we could scale our application automatically and seamlessly in response to the real-time workload, without the overhead of managing servers or clusters. As a result, we achieved a resilient architecture that can dynamically adapt to changing loads, which was crucial for meeting our performance benchmarks.

Heartbeat Challenge:

Another technical hurdle was implementing the OCPP 1.5 protocol’s requirement for regular heartbeat messages. The CPMS expected to receive heartbeat signals from each connected charge point to confirm their operational status. However, the client’s controllers do not inherently send these heartbeat messages.

Solution with Lambda Scheduler:

To overcome this, we designed a Lambda scheduler that triggers AWS Lambda functions at predefined intervals – in this case, every 10 minutes. These functions generate and dispatch heartbeat messages to the CPMS for all connected charge points. By leveraging AWS Lambda, we capitalized on its ability to run code in response to events and automatically manage the compute resources, ensuring that heartbeat messages are sent reliably without the need for a persistent, always-on server process.

Benefits of Our Approach:

The combined use of AWS Fargate and AWS Lambda allowed us to create a system that is not only highly scalable but also cost-efficient, as we pay only for the compute time we consume. This architecture has eliminated the complexity of scaling, managing, and operating server instances, leading to a robust solution capable of supporting our extensive network of electric vehicle charging stations.

Through these strategic choices, we succeeded in designing a system that stands at the forefront of operational efficiency and reliability, fully embracing the cloud-native principles that will carry our solution into the future of energy management and smart grid interoperability.

Can you discuss any innovative or creative solutions that were developed during this project?

During this project, a creative solution that stands out is the implementation of a heartbeat scheduler. This component is a lambda function programmed to activate at ten-minute intervals. Its primary role is to retrieve the list of charge points from the device mapping table in DynamoDB. Once it has this information, it then proceeds to send OCPP heartbeat requests to the CPMS, but only for those charge points that are marked as actually connected. This feature ensures constant monitoring and maintenance of the communication link between the charge points and the CPMS, enhancing system reliability and performance despite the native controller not supporting it.

What was the process for identifying and prioritizing the features that would deliver the most value to the client?

We focused on the following to prioritize the features that would deliver most value:

Understanding the client needs: We started by thoroughly understanding the client’s requirements, objectives, and vision for the system. It was very technical, so we engaged in detailed discussions and follow up meetings so that we could gain a clear understanding of what the client wanted to achieve.
Feature Brainstorming: Internally, within our team we conducted brainstorming sessions to generate a comprehensive list of functionalities that could be included in the system.
Prioritization Framework: We used the prioritization framework “MoSCoW” method (Must-have, Should-have, Could-have, Won’t have) to define the scope of the MVP of the project.
Iterative Feedback: Since we implemented this project within the Scrum framework, we conducted a Sprint Review (Demo) session in order to show progress, ensure alignment and to gain feedback from the client.
Continuous Refinement: Prioritization is not a one-time activity but an ongoing process throughout the software development lifecycle. We continuously reviewed and refined the feature priorities based on changing business priorities, stakeholder feedback, and newfound insights.

How did you manage project timelines and ensure the project stayed on track?

On top of the technical challenges, this project also had a tight delivery deadline. Managing the timeline and ensuring that the project stayed on track was crucial. We followed few steps during the project lifecycle:

Initial planning: After the gathering of initial requirements, we broke down the scope into smaller tasks and identified dependencies between them and allocated people accordingly.
Create a roadmap (Timeline): We developed a detailed project scheduled with all user stories and deadlines. We set realistic deadlines considering the factors like the team capacity, complexity of the tasks and potential risks and dependencies.
Regular monitoring: Regular monitoring was done as part of our scrum way of working, tracking the regular progress on our daily scrum meetings and communicating that with the client accordingly.
Regular reviews: We conducted regular reviews of the project timeline and progress after each sprint. We used those reviews to identify areas where improvements can be made and adjust the schedule accordingly.

What was your approach to testing and quality assurance throughout the project?

We are following different stages in Metergram to effectively manage testing and quality assurance through the whole project lifecycle. For this project in particular, in order to deliver a high-quality project it came down to:

QA involvement: Our QA engineer was introduced to the project from the very beginning so that she was involved from the early analysis phase.
Test Planning and Strategy: A comprehensive test plan was created at the beginning of the project, outlining the test objectives, scope, people, timeline and methodologies.
Comprehensive Test coverage: We executed testing strategies on all levels of the application, including units testing, integration testing, system testing and acceptance testing.
Defect Tracking and Management: We used the Jira management tool to log, prioritize and manage defects identified during the testing. This is also where we tracked improvements and bugs, once in production.

What has been the response from users and stakeholders to the project?

Following the launch of the project’s MVP, we actively monitored the live environment and gathered feedback. We are pleased to report that our scaling strategies and performance focus proved highly effective, resulting in stable system operations despite high load at times. The client was very satisfied with our delivery within such a short timeframe, which enabled them to achieve their business goals related to their charger deployment roadmap.

What were some key lessons learned from this project that could be applied to future endeavors?

Reflection on key lessons learned is essential for continuous improvement. We identified some lessons learned that the team drew from their experience. The key points added from the team were:

Clear requirements gathering: This crucial phase helped the team to have clear requirements at the beginning of the project to avoid misunderstandings and scope creep later on.
Effective Communication: Regular updates, status reports and meetings helped everyone to deliver the project effectively and on time.
Agile Adaptability: We are embracing agile methodologies to facilitate flexibility and adaptability in response to the changing requirements of priorities.
Scalability and performance: We worked on gathering the scalability and performance requirements early in the project to avoid performance bottlenecks or scalability issues as the user base grows.
Continuous improvement: And at the end, of course we are encouraging a culture of continuous improvement by doing sprint retrospectives to reflect on successes, challenges and areas for improvements.