Automate Google Form Submissions With Selenium, AWS Lambda & Amazon EventBridge

Photo by David Clode on Unsplash

Recently, I started noticing that I had to fill up and submit Google Forms on a weekly basis. As someone who loves automating mundane tasks, I quickly got to developing a solution.

At a high level, I settled on creating a python script. There were two routes that I could take: sending a POST request with the correct parameters to https://docs.google.com/forms/d/e/[form-id]/formResponse or using Selenium to mock filling and submitting the form. I decided to go with using Selenium since I had not used it before. Also, I will be using the 2Captcha API for solving reCaptcha if needed on the Google Form (I needed it in this case).

As a Salesforce Developer, I recently had the opportunity to integrate two huge cloud ecosystems: AWS and Salesforce. One of the problems encountered in the project was dealing with Lambda cold starts. A solution would be to create an Amazon EventBridge rule that will warm the Lambda at specific time intervals. Having an understanding of both services, I decided to use them to run the python script weekly.

Simple architecture diagram for form automation

Scripting the Lambda Handler

Let’s start by ensuring that you have AWS Toolkit on VSCode installed. I will be using VSCode for this walkthrough. Next, follow these steps: use the command AWS: Create Lambda SAM Application > python3.6 > AWS SAM Hello World > [YOUR CURRENT FOLDER] > form-automation.

Install requests and selenium to your local environment and import the following to app.py.

Import statements in app.py

I proceeded to code my constants. For the date, I wanted it to display the date for Sunday whenever I ran my Lambda on Saturday. For DATA, there were three different input fields: email, name, and contact number. The MODE ensures that only when it is PROD will the form submit. The other constants will be assigned via environment variables which I will cover later.

Constants for app.py

I coded the options for the Chrome driver, instantiated it, and called upon the Google Form URL.

Selenium options and instantiation for app.py

After, I created a function with the definition auto_fill_form(i) whereby i relates to the position of the lists in DATA. The first part of the function searched and populated form inputs.

Handling input values on the Google Form

For Google Forms with reCaptcha implemented, I used the 2Captcha API. Basically, you would find the data-sitekey attribute for the element with id recaptcha and send it to the API. The API would then respond with a task id. This task id has to be periodically checked for task completion. On completion, capture the results and populate the textarea with the id g-recaptcha-response.

Implementing Captcha2 API

To end off the function, I enabled the form to be submitted if the MODE was PROD. A reset will then take place for the following user.

Form submission and reset

For the Lambda handler, I created a for-loop to loop through the data and call upon the auto_fill_form function.

Lambda handler function to be executed

Configuring the CloudFormation Template

Before I deployed the SAM Application to AWS, I configured the template.yaml. Under Resources > FormAutomationFunction > Environment > Variables, I replace the values that start with “Enter” with my own. After I finished testing, I changed the MODE to PROD.

Configuration for template.yaml

AWS Setup

After deploying the template, I proceeded to attaching the Chrome Driver Lambda Layer. You can find a copy of the .zip file in my repo (Leave a star if you found it useful!).

Adding the Chrome Driver Lambda Layer

I deployed the template.yaml and ensured that the Chrome Driver Lambda layer was attached to the function.

Layer setting for the Lambda function

Next, I navigated to Amazon EventBridge and set a rule to trigger the Lambda at the start of every Saturday with the cron expression cron(4 0 ? * SAT *). Since I live in Toronto, you might need to set the cron expression according to your time zone.

Conclusion

This project was fun to plan out and develop. I enjoyed learning more about automation scripts with Selenium and deploying it to AWS. Now, I can go about my day with one less task in mind!

Note that the input requirements for every Google Form would be different. Repurpose the code from this walkthrough accordingly. Here’s the GitHub repo for those interested: https://github.com/BryanWZC/Form-Automation-Lambda.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Bryan W.

Bryan W.

I have a passion for coding | Salesforce Developer