BUSINESS PROBLEM
Our client was facing a big data challenge, requiring downloading large amounts of data from various web applications. This included sensitive information and required use of secret credentials. This used to be performed manually which would sometimes cause manual errors, was a slow process, and have reliability issues
SOLUTION
We started by understanding complete business requirements- Built an initial POC to ensure that the business requirements have been correctly identified- Used tools like Python, Celery, Selenium, Playwright, FastAPI, and Docker to build a distributed UI automation application, that will perform this entire operation on its own with limited user inputs- Used custom Vault software to store secrets safely and securely in a HIPPA compliant manner- Minimal use of external services, in order to reduce external dependency- Since the application is fully custom, it has minimal cost and operation overhead
IMPACT
Reduced workload on the operations team by saving over 300 man-hours per month – The process saw a 10x speed increase with parallel and automated processing – Reduced human errors with automation and process validations – Added advanced state management and analytics in order to track the entire automation process and get clear insights into the outputs
