Tag Archives: REST API

I built a Google Forms Toolkit Library for .NET!

Watch me building the soon-to-be-famous (lol) my GoogleFormsToolkitLibrary for .NET C#, a nuget library that will help you easily access your Google Forms programmatically in dotnet, letting you load Field Question data, Submit Form data and so on! 😉

In my previous blog article, Programmatically access your complete Google Forms skeleton! where I shared about my adventure into working out a little hack to access your Google Forms page content programatically, I realized it was a solid implementation, and could be used with almost all kinds of openly accessible Google Forms.

So following up on that, this time I will be building a complete .NET library that will consist of all the awesome hacks and tricks that I built around playing with Google Forms myself, which I have also continuously written blog articles as well!

Backstory…

So far we have come across…

And then in my previous blog article, Programmatically access your complete Google Forms skeleton! I shared how we retrieving the following data on a given Google Form extensively,

  • Google Form Title, Description, Form ID
  • List of Question Fields

And in each Question Field,

  • Question Field Text
  • Question Type
  • If submitting Answer is mandatory or not
  • Available answer list (Multiple answer selection)
  • Question Field Identifier (Field Id) or Answer Submission ID

Then we analyzed the data structure of the FB_PUBLIC_LOAD_DATA_ script that we scraped out of the rendered HTML content, and even managed to parse it into a meaningful data structure which could be mapped to the data we are looking for.

So using that knowledge and expertise, let’s step up the game and build a solid library tool, which we can reuse easily in anywhere to access our Google Forms programmatically to perform all kinds of awesomeness as we wish! 😉

However if you haven’t gone through my previous blog articles in this Google Forms Hacks series, please do so before continuing to avoid any confusion, since I might not be diving into all the details.

Behold GoogleFormsToolkitLibrary v1!

So this library that I’m building will be on .NET Standards, which will allow you to use it in almost any kind of a .NET project. From Console apps, Desktop, Web and even to Mobile Apps in Xamarin!

It will provide you with the following awesome features!

  • Load information on your Google Form
  • Load Question Field data on your Google Form
  • Submit Form data to your Google Form

At least for now, I will be adding more features as I go along my journey of hacking around Google Forms! 😉

I will be publishing this library to Nuget, so that you can easily grab it into your .NET projects, also let’s add some fancy Test Driven goodness to it, using xUnit Tests! 😀

Project Solution Set up…

So I created a .NET Stanard version 2.0 library in Visual Studio, naming it GoogleFormsToolkitLibrary of course! 😉 The I added a Test folder into the Solution which will hold the xUnit Unit Test project, which I named .Tests!

Then don’t forget to add the HTMLAgilityPack and Newtonsoft.Json nugets to the Library solution which will allow us to scrape out the HTML content in a Google Form and use the Json content parsing logic!

Now the basic set up is done, let’s move on to the code!

Let the coding begin…

We need to model the data objects that we’re going to handle when it comes to the perspective of a Google Form. Basically following are the main entities of a Google Form as far as we have recognized in our previous blog articles!

  • A Google Form
  • A Google Form Question Field
  • The Google Form Field Type

Then as of the functionality I would be adding two public methods that we can call upon to execute.

  • Load Google Form Structure, data, and form fields
  • Submit form data to a given Google Form

Crafting the Models…

So we let’s start by building Model classes that will represent the entities we need.

GoogleFormsFieldTypeEnum

Let’s start with the simplest Model we could build that is the Google Form Field Type object, which will represent the type of a given Question Field. I basically described in detail about this entity and how I build the entity model myself through trial and error in my previous blog article:

Let’s create an enum model class giving it the name  GoogleFormsFieldTypeEnum.

We’re adding the Description tags to hold the human readable value of each type.

GoogleFormField

Now this fella right here is a big deal, that is the Google Form Question Field object, which will represent a single Question Field in a given Google Form. A whole list of these fields comprises in every Google Form.

Let’s create a model class giving it the name  GoogleFormField.

As you can see we have created properties inside this model that will hold all the values that we discussed previously, that could consist in a given Google Form Question Field.

GoogleForm

Now here’s the top entity, representing a whole Google Form page content. Let’s create a model class giving it the name  GoogleForm.

Pretty straight forward set of properties that would usually contain in any Google Form as you can see above. Now make sure to add those Model classes to the Models directory in your project.

Alrighto, we got the model classes done, then let’s move on to the actual functional coding!

Implementing the core features…

We’re going add two main functionality to this library, for now as following! 😉

LoadGoogleFormStructureAsync

Now this method will intake a string parameter that will carry your Google Forms link, and it will return an object of type GoogleForm that we define above. Which obviously contains the given Google Form’s generic information and Question Field list data including Question Type, Answer Options, Submission Id, etc. Now that’s quite a handful eh! 😉

Task<GoogleForm> 
LoadGoogleFormStructureAsync(string yourGoogleFormsUrl)

So this fella will make Http Async call to load the rendered HTML of the Google Form into memory, run through my magical data scraping algorithm, build the GoogleForm object and return it! Sounds pretty simple but don’t let the implementation confuse you, take a look here! 😉

https://gist.github.com/UdaraAlwis/dcb473f9f07d5024376f1289c1b08ace

Please feel free to take a look at the full implementation on Gist link.

This method is more of an extension of the ScrapeOffFormSkeletonFromGoogleFormsAsync() method that I shared with you all in my last blog post, which I gotten into in depth details about explaining each step of the code.

So to reduce repetition I’m not going to repeat the same here, rather I would share the important bits to explain that are improvements on top of the previous implementation.

One of the differences I would say is that now we’re injecting the data that we scraped out into our model objects, as you can see we’re initializing out GoogleForm and GoogleFormField data objects here, and then loading the GoogleForm data object values.

Then inside the loop we’re loading the GoogleFormField object values as well, and then each object will then be added to the QuestionFormFieldList property of the GoogleForm. That’s pretty much it! 😉

SubmitToGoogleFormAsync

This method will intake a string parameter that will carry your Google Forms link, and  a Dictionary object containing the Form field answer submission id and answer values mapping. As a result it will return a boolean value denoting the success or failure of the submission task.

Task<bool> SubmitToGoogleFormAsync
(string yourGoogleFormsUrl, Dictionary<string, string> formData)

It will make a Http Async call the Google Forms REST API endpoint with the given data and await for the response code, upon receiving the response it will return true or false based on it.

https://gist.github.com/UdaraAlwis/8d547f68ae3f629d40b2184f51acd43e

Please feel free to take a look at the full implementation on Gist link.

This methods is actually an improvement of one of my previous posts, You may RESTfully submit to your Google Forms… where I explain in depth how I hacked around to figure this out and what each line of code is meant to handle. Please look into that article if you’re keep for more details.

We’re basically looking for the 200 StatusCode value that determines successful submission to the Google Forms REST API endpoint.

Now our Core Library is ready to go!

Unit Test it yo!

Yep but its not done until we implement a proper set of test cases isn’t it! So let’s add some Unit Tests that will test the functionalities that we built into our Library.

Like I mentioned in the Project set up, we have added a xUnit Unit Test project into the solution, and make sure to add a reference of our Library project GoogleFormsToolkitLibrary into it.

Let’s start by adding a Tests.cs class to it, which we will use to implement the tests. For now, I will add two test cases,

RetrieveGoogleFormStructure_Success

This will test for LoadGoogleFormStructureAsync() feature and make sure we get a valid response and the expected data of the given Google Form page.

SubmitDataToGoogleForm_Success

Now this case will test for the SubmitToGoogleFormAsync() feature and make sure we get a successful boolean response, with the given Google Form link, and the form data dictionary object.

Oh let’s not forget to make sure they’re passing in the Test Runner! 😀 lol

All good to go it seems! for now at least lol 😛

This is really a great practice because if I ever break anything in the code during any new implementation or existing modifications, I would be able to notice it here in the tests.

Nugetting it!

Let’s not forget to publish the beauty to Nuget eh! Let’s make sure all the nuget properties are added to the package before uploading…

And you may find it on nuget right now!

Nuget: nuget.org/packages/GoogleFormsToolkitLibrary/

This whole project is open sourced and published to Github as well if you’re interested in looking at the code, or track the improvements I’m adding in the future!

Github: github.com/GoogleFormsToolkitLibrary

So that’s done!

Let’s try it out!

Now the beauty of this is that you can easily add this to any of your .NET projects as the whole library is based on .NET Standard.

Go ahead and add it to your .NET project from Nuget using either the Package Manager Console or the Nuget Package Manager.

Install-Package GoogleFormsToolkitLibrary

Use of GoogleFormsToolkitLibrary is pretty simple, just instantiate it and call upon the method you wanna use.

// Retrieve the structure of my sample Google Forms page
// https://docs.google.com/forms/d/e/1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA/viewform

var googleFormLink =
"https://docs.google.com/forms/d/e/" +
"1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA" +
"/viewform";

var googleFormsToolkitLibrary = new GoogleFormsToolkitLibrary();
var result = await googleFormsToolkitLibrary.
                    LoadGoogleFormStructureAsync(googleFormLink);

 

That should work like a charm! Make sure to pass your Google Form’s link properly and make sure it is openly accessible, you should be able to see the magic pretty easily!

// Submit data to my sample Google Forms page
// https://docs.google.com/forms/d/e/1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA/viewform

var googleFormLink =
	"https://docs.google.com/forms/d/e/" +
	"1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA" +
	"/formResponse";

Dictionary<string,string> formData = new Dictionary<string, string>
{
	// Question Field 1
	{"entry.1277095329", "Moon Rockets Launching"}, 

	// Question Field 2
	{"entry.995005981","Banana Plums"},

	// Question Field 3
	{"entry.1155533672","Monkeys with hoodies"},

	// Question Field 4
	{"entry.1579749043","Jumping Apples"},

	// Question Field 5
	{"entry.815399500_year","2019"},
	{"entry.815399500_month","11"},
	{"entry.815399500_day","11"},

	// Question Field 6
	{"entry.940653577_hour","04"},
	{"entry.940653577_minute","12"},
};

var googleFormsToolkitLibrary = new GoogleFormsToolkitLibrary();
var result = await googleFormsToolkitLibrary
                .SubmitToGoogleFormAsync(googleFormLink, formData);

 

The above should nicely respond with a successful true value! 😉

Something to keep in mind here is that you need to make sure you’re setting the correct Field Answer Submission ID properly, and when it comes to Multiple Answer selection fields, make sure the provided answers matches the available list of answers. Date Time fields should be carefully treated with their additional year,month,day suffixed fields and hour,minute respectively. Just go through my past few blog posts on Google Forms hacking, you’ll see for yourself!

What’s next?

Well building this and publishing it openly is just the first step, I’m going to continue building this further adding more features and performance improvements.

Then I will be building some demo Client apps which will use this library to implement some cool features… 😀

So keep an eye out!

There you have it, how I built a Google Forms Toolkit Library for .NET!

Share the love! 😀 Cheers!

You may RESTfully submit to your Google Forms…

You wanna submit responses to your Google Forms in a REST-ful API call, or rather programmatically in code or easily from a Postman-like tool? Then you’re welcome to stick around here… 😉

So you remember my last post on my journey of hacking around Google Forms, trying to be a smart-ass eh! Let’s auto fill Google Forms with URL parameters… Oh yeah that one right there, well that was just the tip of the ice berg of me playing around with Google Forms! Let me share the next set of cool tricks I figured out here! 😀

This little trick of submitting data RESTfully to your Google form, could become very handy if you wanted to build your own improved custom UI for submitting data to your Google Form, along with your own validations for the fields or even to quickly populate a bunch of sample data from your Form for experimental reason. Them awesome possibilities are endless! 😉

Well.. Google Forms with RESTful ?!?

So during my adventures into messing around with Google Forms, I figured out that we can submit data into our Google Forms using their REST API endpoint! So how cool is that eh, we can directly post data into our form RESTfully, from whatever the medium you prefer, programmatically, or Postman like tool! 😉

So in this post lemme share that cool trickery bits with you…

Let the hack begin…

Now keep in mind unlike the last post, this is a bit advanced trick which requires some experience on HTML and web development, well it could easily help at least.

We’re gonna get the REST endpoint source of our Google Form, package our question-field-answer data into a request object and submit it to the REST endpoint directly, using Postman or Programmatically in code.

Now for this post also let’s use the same sample questionnaire Google Form that I created for last post’s demo.

https://docs.google.com/forms/d/e/1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA/viewform

Little sneak peak:

So now we got a Google Form, let’s start the little hack by finding the field IDs in the form…

Yes, you still gotta hook up the fields!

Remember in my last post I explained about the Fields in our Google Form having unique identifiers (or IDs) that we can use to attach data into for submission? well you still need em! 😛

Now for this you could still use the methods we discussed in the previous post to get the list of ID of the fields in your Google Form, but this time I’ll introduce some easier ways, since we’re moving to a bit advance trick…

Hooking up to the fields and Endpoint…

Keep in mind this requires a little bit experience in web dev! 😉 Basically we’re going to get the list of Field IDs by tracing the submission request call in our Google Form, which will also help us figure out the REST endpoint link.

So open up your Google Form in Chrome browser and open up developer tools by using the browser menu or on Windows click “Ctrl+Shift+I keys” in the keyboard.

Now to make the magic work, go to “Network” tab in the menu list which will allow us monitor the network trace that’s going to be sent from browser to Google Form submission REST endpoint.

Next, you need to fill up all the question fields in your Google Form and hit submit button. Carefully watch what happens in the developer console!

Yep a whole bunch of logs pops up, which shows you the traces of all the network calls that occurred in the last “Submit” button click. And in there the most important request log is the “formResponse” log as you seen above.

Click on formResponse log which will bring up all the details on it.

Now this is quite awesome, it will show you in 4 separate sections all the details about the Google Form submission data endpoint that just occurred.

https://docs.google.com/forms/d/e/1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA/formResponse

The Request URL is the endpoint we’re going to be using to submit our form data and the Form Data section is where you’ll find the list of field identifiers of your Google Form.

Now that your holy grail of list of field identifiers in bulk. So go ahead, highlight that chunk of text and copy it up to some notepad to be used later.

Now if you noticed the ID with the “entry.1155533672_sentinel” is something that you can ignore, since its a repeated field coming from the Check box question field in your Google Form!

Just like that you can easily extract the list of IDs of the fields in your Google Form! 😀

entry.1277095329: Bibimbap Turtles
entry.995005981: Banana Plums
entry.1155533672: Dogs with hats
entry.1579749043: Jumping Apples
entry.815399500_year: 2019
entry.815399500_month: 11
entry.815399500_day: 11
entry.940653577_hour: 00
entry.940653577_minute: 30

Now that’s the form data sample from my Google Form! 😉

Shove it up into a Postman!

Or any of the web dev tools you prefer to make a simple REST api call. Up to you! lol. Let’s create a POST call with our Google Forms submission API endpoint which we retrieved in the above step.

https://docs.google.com/forms/d/e/1FAIpQLSeuZiyN-uQBbmmSLxT81xGUfgjMQpUFyJ4D7r-0zjegTy_0HA/formResponse

Actually this URL you could easily make up using your Google Form publish url, just by replacing viewform with formResponse suffix.

So make sure to add the Body parameters of type x-www-form-urlencoded, and list out all the question field IDs and their values you’d like to inject in to the submission. Since then you need to apply header Content-Type as application/x-www-form-urlencoded which will enable our body parameters object.

Assuming you have set up all the body form fields properly, let’s fire it up! 😀

Fire it up!

Let’s go ahead and execute the REST posting! Hit “Send“, and bam!

You should get a successful 200 Status Code response with the above success message “Your Response has been recorded.” inside a whole bunch of HTML content and if you move to the “Preview” tab, you can see how the rendered UI being returned as well.

Now let’s say if you missed adding any specific field in the request body, that was already marked as “Required” in your Google Forms template, and you had hit “Send”. In that case it would return a bad request 400 Status Code with the error in HTML content, “This is a required question”, or with whatever the custom error message you configured your Google Form with.

Yeah you can even view in the Preview tab with the rendered HTML content.

Pretty neat eh! the same way it would behave in a browser environment you can duplicate in a RESTful environment such as Postman! 😀

Now let’s see how easy it is to push that into some code and execute this programatically!

Shove it up into a Code snippet!

Alright let’s shove that into a simple C# snippet where we could POST a simple HTTP request with the body parameters of our Google Form! Basically replicating the same execution as Postman you saw above! 😀

Above we’re using a simple dotnet HttpClient to execute our Google Form submission REST post call, by adding the body values dictionary into the request.

And then we’re printing the Status Code and the HTTP content response we get.

Hit F5!

If you hit F5 on the above code in Visual Studio, you should get the following.

We are successfully submitting our Google Form data to the REST endpoint and getting the success code, along with the usual HTML content, same as how got in Postman. 😉

Now let’s say if something went wrong, or you missed a required field in your request body,

It will show up the same error handling we got in Postman, as 400 Bad Request! And if you dive into the HTML content, you can see how the error message was also delivered back.

So now you know how to programmatically submit data to your Google Forms in code! 😉

Imagination is the limit yol! 😉

Well… That’s it!

It’s quite straightforward how Google has built these forms in such a simple manner for you to handle them easily as you wish. Kudos Google!

Now keep in mind all these are simple hacks and tricks derived by careful observation of html code and network traffic, and we do not have precise control whether Google will change these format and rendering patterns in future, so you gotta keep an eye out if you’re planning to use these hacks for a long term solid implementation.
My suggestion would be to write up a series of Test cases (TDD yo!) which would test for the above process flows to make sure they’re working as expected and notify you in case of any changes from Google. 😉

You can do all kinds of cool stuff with it! So there you have it, how you may RESTfully submit to your Google Forms!

Share the love! 😀 Cheers!