XAMVVM-03 SpecFlow BDD UnitTesting with Xamarin.Forms!

Let’s explore the possibilities of the awesome SpecFlow Behavior Driven Testing Framework with Xamarin.Forms!

I’m a huge enthusiast of writing clean and highly testable code, and in that pursuit, I believe Test Driven Development for the win! 😉 Oh also, UnitTesting for life yo! 😛

Problem with just UnitTests!

They’re very useful when we need to sure there’s nothing broken in the logical code, but the only problem is that, they’re way too technical. They don’t really make much of a sense to a non-technical person, since its all code and logic based and the out put is not really human friendly (for a non techie).

Why Human-Friendly testing?

Software Development process is not just a technical dev fellow write the code and deploy to the user process, its way more complicated and involves a lot of parties. Whether you like it or not modern day Software are purely driven by Business Requirements, which includes a lot of non-technical parties into the Software Development cycle. Business fellows, Testers, Analysts and Developers all come together to develop systems nowadays.

This is why we need a more human-friendly and human-readable way of executing the testing process, that can be shared among all the stake holders and could make sense to them from the beginning to the end of the SDLC.

This is where Behavior Driven Development comes into play, it defines a common ground that can be shared among all the stakeholders that are involved in a development of a software, from a techie to a non-techie, from Developers, Testers, QA Engineers, to Business Analysts and even directly to the Customer.

Behavior Driven Development!

BDD tests are written in a way that it is easily understood by everyone, even before the development life cycle begins, while making sure the requirements are specific and well defined. Every single feature is broken into scenarios with specific test steps that make sure behavior is executed as expected, thus behavior driven development 😉

These tests are written in a format called Gherkin, is a business readable and domain specific language created for writing BDD tests. Here’s a little example:

“Feature” tag explains the functionality of a feature, and a single Feature could contain many “Scenarios” that tests the functionalities related to that Feature. Each scenario has steps where you combine the actions using the Given, And, When, Then keywords. 🙂 Simple as that eh! 😀

We could also interpret it almost same as Test Driven Development based UnitTesting but with an extra awesome layer of icing called Behavior Driven Development!

Cucumber and SpecFlow!

Now in order to actually implement these BDD Tests using Gherkin syntaxes, we used a framework called Cucumber, which is a Ruby based framework!

But when it comes to .NET we have an awesome wrapper framework for Cucumber called SpecFlow, that provides us with all the goodness of BDD Testing.

SpecFlow is open source and provided under a BSD license. As part of the Cucumber family, SpecFlow uses the official Gherkin parser and supports the .NET framework, Xamarin and Mono.

SpecFlow seamlessly integrates with Visual Studio and provides all the functionality and features required to carry out your BDD life cycle.

Following a Pattern…

Although SpecFlow constructs its Tests based on Features, you could organize your Test cases as you wish, either based on Pages, Application Flows, or even Features themselves.

Page based pattern – based on behavior and functionalities available in each page in the application, such as example Home Page, Shopping Cart Page, Payment Page, and etc…

Application Flows – based on each individual feature flow of the application, such as Login Flow, Order Flow, Payment Flow, and etc…

Features – based on each individual feature flow of the application, such as Login Feature, Order Feature, Payment Feature, and etc… (quite a bit similar to App Flows)

You could also mix and match between those pattern, based on your requirements or the complexity of your application.

SpecFlow with Xamarin.Forms!

Since SpecFlow bring the beauty of Gherkin-BDD  goodness to .NET we can directly use it in our Xamarin.Forms project solution as well any .NET project solution.

We’d basically be setting up our Xamarin.Forms project with a basic UnitTest project, and we’d add SpecFlow set up and configuration on top of it. In one of my previous blog posts I’ve explained how to set up Xamarin.Forms with UnitTesting project XAMVVM-02 UnitTesting with Xamarin.Forms!, hence I won’t be getting into that details, but rather focus on the set up of SpecFlow tid bits. So I would recommend taking a sneak peek at it before continuing here…

In a gist, we’ll create a UnitTesting project with SpecFlow set up and add our Xamarin.Forms project reference to it. Then we add Xamarin.Forms.Mocks library, for mocking Xamarin.Forms runtime during our test executions. Then we set up our Test App instantiation, with all the required dependencies, which is going to be used across all our test cases.

Once our Test project is ready, instead of writing usual unit test cases, we’d be writing SpecFlow test cases using Spec Feature files. Those Feature files have a C# code behind that’s bound to it which is where we’ll be implementing our test code logic and assertion.

That’s pretty much it! pretty straight forward eh! 😉 Let’s try it out…

Let’s get started…

So before we begin you need to make sure you have the SpecFlow for VS extension installed, if you haven’t, please install from the VS Marketplace which is completely free.

SpecFlow provides a wonderful documentation about how to get started: https://specflow.org/getting-started/ but it delivers for generic .NET projects, not specific for Xamarin. But I would still encourage you take a look through it first just to get some clarity!

Demo Overview!

Now that’s out of the way, let me tell you how I’m going to showcase the use of BDD SpecFlow UnitTesting in your Xamarin.Forms project.

Just for showcasing of this awesomeness, I’ve prepared a simple Xamarin.Forms project application, where you can create simple notes, attach location data, attach created date time values and store them locally, allowing you to view them at any time in a list of items. Quite a simple text-pad-like app.

I’ve named this project solution as, XFTextpadApp and it uses Prism as the MVVM framework.

As you can see above is a view of the project structure with the complete MVVM set up. Let me also share a bit of showcasing of the app and its features to give a more in depth context.

Here’s a little peek into features of this demo Xamarin.Forms Textpad app running on iOS!

In the Home page you can view the List of Texts you have saved. By clicking on a Text item in the Home page list you can view that Text item in detail in the View Text page. Then once you click the “New Text” Button it will take you to the New Text page where you can create and save new Texts…

In the New Text page you can create new Texts with a Title and Description of your Text and click “Save” button to save it with the Location and Time data attached to it.

Then you will be navigated back to Home page. In the Home page Text list you can delete saved Text by using the context menu as well…

Thanks to Xamarin.Forms we can deploy it directly to iOS, Android, and UWP as well! 😉

Now the App project is ready, we need to add Test project to it.

Hands on time!

Let’s begin by adding the Test project to our Xamarin.Forms project, create a new folder in the solution, named as “Tests” and then right click on it -> Add -> Add New Project

Now this is where the choice is absolutely up to you, as to which testing framework to choose from. Just for this demo, I’m gonna use MSTest Test framework, hence I will select MSTest Test Project (.NET Core) template and create our test project…

I will be naming the test project as, XFTextpadApp.SpecFlowTests making sure to stick to the standard naming practices! You might want to pay some attention to the Project Path there as well, where as to make sure we set the path to be inside the “Tests” folder that we created earlier.

Setting up SpecFlow!

Next let’s set up SpecFlow in our Test project, by adding the following SpecFlow nugets using the Nuget Package Manager!

• SpecFlow
• SpecFlow.MsTest
• SpecFlow.Tool.MsBuild.Generation

Make sure to add them all in same version for compatibility reasons, as shown below.

One thing to note here that, I’ve added SpecFlow.MsTest package, while the official SpecFlow documentation recommends adding SpecRun.SpecFlow. The reason is, in order to use their recommended test runner, SpecFlow+ Runner (which is configured by SpecRun.SpecFlow), you need to have a registered SpecFlow account. Therefore I’m using the SpecFlow.MsTest test runner engine instead! 😉

If you need to access the full set of enterprise-grade large scale features of SpecFlow+, then you need to use their SpecFlow+ Runner.

Also if you’re using an alternate test framework like NUnit, or xUnit, then you need to use their own counterpart test runners here, such as SpecFlow.NUnit or SpecFlow.xUnit packages and so on. For more information here: https://specflow.org/documentation/Unit-Test-Providers/

Setting up the rest…

In order to complete rest of the set up, let’s add a reference of the host XFTextpadApp .NET standard project into our XFTextpadApp.SpecFlowTests project. Then we’re going to add our Xamarin.Forms Mocking library, Xamarin.Forms.Mocks.

Just to make sure we write some beautiful test code, let’s add Shouldly nuget as well. This is more of a personal choice 😉 or if you prefer you could use the default assertions of MSTest (MsTest.Framework.Assert), or which ever the test framework you have chosen to set up.

If all went well then you should see the Test project’s references set up as above. Now that’s out of the way, next on let’s get into coding!

Create app instance…

You could directly use the existing App class instance inherited from the Xamarin.Forms project within our Test project environment, or you can construct your own custom App instance that’s catered for the Test environment, which is what we’re going to do in this step…

Something to note here is that, the way you construct your Test App instance is completely dependent on how you have constructed your Xamarin.Forms application, so try to duplicate it as much as possible.

Let create a new class called TestApp in our XFTextpadApp.SpecFlowTests project root.

public class TestApp : PrismApplication
{
    public TestApp() : this(null) { }
 
    public TestApp(IPlatformInitializer initializer) : base(initializer) { }
 
    protected override async void OnInitialized()
    {
        ...
         
        await NavigationService.NavigateAsync("NavigationPage/HomePage");
    }
 
    protected override void RegisterTypes(IContainerRegistry containerRegistry)
    {
        containerRegistry.RegisterForNavigation<HomePage>();
        ...
 
        containerRegistry.GetContainer().RegisterType<HomePageViewModel>(new Unity.Lifetime.ContainerControlledLifetimeManager());
        ...
    }
}

Now, you need to make sure to register all your Pages, ViewModels and Services accordingly. Also notice that how I have set up the default start up navigation in the OnInitialized() method according to Prism.

Next the Hooks…

This is more like the helper class that we’ll be using to set up the necessary hooks for our SpecFlow test run time environment, such as instantiating the App instance object, and initialising MockForms. Here we’re going to keep a public static object of our App instance, which we will use all across out tests cases. Let’s create a class called TestHooks…

[Binding]
public class TestHooks
{
	public static TestApp App { get; private set; }
	
	[BeforeScenario]
	public void BeforeScenario()
	{
		Xamarin.Forms.Mocks.MockForms.Init();

		App = new TestApp();
	}

	[AfterScenario]
	public async void AfterScenario()
	{
		// After Scenario steps

		// 1. Clear App data in memory
		Application.Current.Properties.Clear();
		await Application.Current.SavePropertiesAsync();

		// ...
	}
}

SpecFlow provides Before and After Scenario handlers which we can use to perform operations as we see fit, hence I’m using [BeforeScenario] attribute with the BeforeScenario() method to initialise MockForms library and the app instance before each scenario. Scenario refers to a single test case, inside of a Feature in SpecFlow. Similarly [AfterScenario] attribute is used for cleaning up operations after scenario execution before. The [Binding] attribute is a marker informing SpecFlow that this class may contain hooks or step definitions, thus enabling the before and after handlers.

SpecFlow provides many different binding hooks like these that you can adopt for handling many different scenarios: https://specflow.org/documentation/Hooks/

You may have noticed the public static TestApp object here, which is the App instance that we’re going to use across all our test cases.

Alright, but we’re not done yet!

Creating the Features and Scenarios!

As I’ve explained before, in SpecFlow we create Features, then inside those Features we create Scenarios, in each Scenario we create Steps, and finally we bind those Steps to Step Definitions in C# code behind! Check out the extended explanation here: https://specflow.org/documentation/Using-Gherkin-Language-in-SpecFlow/

Now that all is set up and ready to go, let’s create our actual Tests. But first create a “Test” folder in the root of our XFTextpadApp.SpecFlowTests project.

If everything is well set up until now, then it should look like the above structure.

In the beginning I explained about following a pattern for organizing your SpecFlow Tests, and for this demo I will be following a Page context based pattern for my Tests.

So let’s consider our Home Page and one of its testable scenario such as,

Scenario:
First time I launch the app
I am Navigated to the Home page
I see an empty list of data
I see a label indicating empty data

Now let’s create our first Feature by, right click on the “Tests” folder -> Add -> New Item -> select SpecFlow in C# Items list -> select SpecFlow Feature File. Let’s name it as HomePage.feature and Add it.

You should see the HomePage.feature file created in your project, with some default SpecFlow content in it. Let’s replace all the default content with the following,

Feature: HomePage

Scenario: App Launching First Time Test
          Given I have launched the app
          Then I am in the Home Page
          And I can see 0 Items in Text List
          And I can see a Label indicating empty Text List

So in here you can see how I’ve defined our Feature as the HomePage, and then created the first Scenario accordingly. There we just created our first Feature and Scenario! 😉

One thing to note here is that, once you type out those steps, they’re highlighted in purple color. This is because those steps are not bound to any steps definition.

Until we create a Step definition file and bind those steps to the C# code behind, they’d stay highlighted in purple. Remember I told you, each test scenario is a combination of Steps and their C# code behind definition?! 😉

So next on, let’s create our Step definition file for our Feature by, right click on the “Tests” folder -> Add -> New Item -> select SpecFlow in C# Items list -> select SpecFlow Step Definition. Let’s name it as HomePageSteps.cs and Add it.

You will see the HomePageSteps.cs class generated with a bunch of templated code. Let’s remove all the auto generated code except for the following bits, with Constructor and private ScenarioContext object.

[Binding]
public sealed class HomePageSteps
{
	private readonly ScenarioContext _context;

	public HomePageSteps(ScenarioContext injectedContext)
	{
		_context = injectedContext;
	}
}

So this is the class where we will be creating our Step definitions for our Feature Scenarios. Notice the [Binding] attribute which marks this class as a candidate for step definitions. The constructor inject ScenarioContext provides you access for passing data between different Scenarios and Steps. More info: https://specflow.org/documentation/scenariocontext/

SpecFlow extension in Visual Studio allows us to generate the C# definitions for the scenario steps that are unbound. So let’s generate the step definitions for our HomePage.feature Feature and its Scenario that we created before.

Open the HomePage.feature file and right click anywhere inside the feature file. Then select the menu option at the bottom, “Generate Steps Definitions“, which will popup the following,

Here you can use the Copy methods to clipboard button and paste these generate C# definitions in your HomePageSteps.cs class file. If you select generate it will generate the whole steps file for you, and ask you to select the folder destination to save the file.

So after generating the step definitions our HomePageSteps.cs class file should be updated as shown below, (I’m show only one definition here in the code)

[Binding]
public sealed class HomePageSteps
{
	...

	[Given(@"I have launched the app")]
	public void GivenIHaveLaunchedTheApp()
	{
		ScenarioContext.Current.Pending();
	}
	
	...
}

By default the generated code will add the ScenarioContext.Current.Pending(); line for marking the step as pending. You can go ahead and remove that in all the steps and let’s complete our Test implementation…

[Binding]
public sealed class HomePageSteps
{
	public TestApp App => TestHooks.App;

	[Given(@"I have launched the app")]
	public void GivenIHaveLaunchedTheApp()
	{
		// Is the app init and running
		App.ShouldNotBeNull();
	}

	[Then(@"I am in the Home Page")]
	public void ThenIAmInTheHomePage()
	{
		var navigationStack = ((NavigationPage)App.MainPage).Navigation.NavigationStack;
		// Am I in the Home page
		navigationStack.Last().BindingContext.GetType().
		    Name.ShouldBe(nameof(HomePageViewModel));
	}

	[Then(@"I can see (.*) Items in Text List")]
	public void ThenICanSeeItemsInTextList(int itemCount)
	{
		// Checking the number of items in the List
		App.Container.Resolve<HomePageViewModel>().
		    TextList.Count.ShouldBe(itemCount);
	}

	[Then(@"I can see a Label indicating empty Text List")]
	public void ThenICanSeeALabelIndicatingEmptyTextList()
	{
		// Check if the Empty ListView Label Displayed
		App.Container.Resolve<HomePageViewModel>().
		    IsEmptyTextList.ShouldBe(true);
	}
}

So as you can see, we’re basically following a mix of ViewModel and UI Element objects to check for the requirement of each step in this code behind test code. Also I’ve taken the liberty to rename the parameter names such as “int itemCount” to my liking! 😉

One more thing you can notice is that, if you take a look at the HomePage.feature file, its Scenario steps will be highlighted in white color as follows, confirming that our Scenario Steps are now bound to C# code behind Step Definitions.

Aha! there you go, a happy happy feature file ready to go! 😀

So we created our HomePage Feature file and Steps files, and they’re sitting in harmony next to each other in our project. What’s next you ask?

As of now, if everything is set up properly, you should see your first Feature Scenario ready and executing in Test Runners without any issue! 😉

Let’s run it yo!

And it passes the test with beautiful green! lol 😀

Yes, more Scenarios!

Yes, like I mentioned before, you can add any number of of Scenarios in a given Feature, so let’s go ahead and add more scenarios to the HomePage Feature!

Feature: HomePage

Scenario: App Launching First Time Test
          ...

Scenario: Navigating to New Text Page Test
          Given I have launched the app
          Then I am in the Home Page
          And I click on "New Text" Button
          Then I am in the New Text Page
        
Scenario: Deleting Text Item Text
          ...

Simply a matter of following the same pattern, and binding to the Step definitions behind in code behind! Go crazy! 😉

Clean up with Sharing…

Now let’s take it up a notch, by sharing our Step Definitions, which will certainly increase the reusability of our test code and reduce repetition. SpecFlow uniquely facilitates this feature where, any given Step Definition is shared and accessible across the same namespace.

Let’s create a new class, SharedSteps.cs which will be used to hold all our common shared step definitions…

[Binding]
public class SharedSteps
{
	public TestApp App => TestHooks.App;

	[Given(@"I have launched the app")]
	public void GivenIHaveLaunchedApp()
	{
		...
	}
	
	[Then(@"I am in the ""(.*)"" Page")]
	public void ThenIAmOnThePage(string pageName)
	{
		...
	}
	
	...
}

We need to add the [Binding] attribute to mark the class for SpecFlow bindings, and then we include all the step definitions we need to share across all our Features under the same Test environment.

Passing Arguments…

As you noticed before, we can pass any number of arguments or parameters in a given step, as either integer or string values. If it’s an integer parameter you can directly pass it as is in the step, but if you want to pass a string parameter, then you need to surround it with upper quotes as shown below…

Scenario: Creating new Text Item Test
          ...
          Then I am in the "Home" Page
          And I can see 0 Items in Text List
          ...
          Then I am in the "NewText" Page
          And I add New "Test Title 1" and "Test Description 1"
          ...

The above is handled in the code behind step definition as follows,

[Then(@"I am in the ""(.*)"" Page")]
public void ThenIAmOnThePage(string pageName)
{
	...
}

[Then(@"I can see (.*) Items in Text List")]
public void ThenICanSeeTextItemInListView(int itemCount)
{
	...
}

[Then(@"I add New ""(.*)"" and ""(.*)""")]
public void ThenIAddNewAnd(string textTitle, string text)
{
	...
}

There you go pretty neat eh! but let’s take it up a notch, where what if we need to pass multiple values for each parameter in the step? Now that’s where Scenario Outline comes to rescue,

Scenario Outline: Creating new Text Item Test
                  ...
                  And I add New "<TextTitle>" and "<Text>"
                  ...

Examples: 
	| TextTitle    | Text               |
	| Test Title 1 | Test Description 1 |
	| Test Title 2 | Test Description 2 |
	| Test Title 3 | Test Description 3 |

Using a Scenario Outline, you can simply provide multiple values for each parameter in the Example table. You can see in the above example, where I’m passing the parameters “TextTitle” and “Text” with multiple values into the Scenario. SpecFlow will handle this during test runtime as separate scenario executions with the dataset in each row. Clever trick! 😉

SpecFlow blog has a full fledge post on this, which I highly recommend you take a look: https://specflow.org/2019/improving-gherkin-specs-with-specflow-step-argument-transformations/

Just like that, SpecFlow is full of awesome features you can utilize build your Tests! 😀

Mock em!

Hence we’re using  Xamrin.Forms.Mock, it should be able to handle all the Xamarin.Forms related behaviors and functionality in our Test environment directly, such as Navigation, UI Elements rendering and so on. If you have any platform specific features, then you need to mock those service instances in your testing environment.

I’ve one a similar case here in my demo with the ILocationService, which has a MockLocationService implementation in the Test environment.

If you need to mock any sample fake data, then you could use an awesome library like moq or any other alternative such as, Faker.net and Bogus which are also popular!

In a scenario where, you have actual API calls going in or out of your app, you shouldn’t use them in your test run time. You need to create mocked Services for those API bound calls, with fake data sets attached to them, as the standard practice for Testing.

Finalizing…

So all of this awesomeness is committed into my github repo:

Github repo:
github.com/UdaraAlwis/XFWithSpecFlow

Since I took the Page based pattern for organising my SpecFlow tests, here’s the set of Features I ended up creating. Each Feature has its own Steps Definition class alongside them. There are multiple Scenarios in each Feature.

Under the Test folder I’ve placed all my SpecFlow Features, but you can restructure this hierarchy as you please, even include your own sub folders containing different sets of Features as well… 😉

You can see the SharedSteps class that I extracted for maintaining shared step definitions. So to sum it up, in my demo below are the SpecFlow Features that represents each page…

• Home Page : HomePageFeature
  • Scenario: App Launching First Time Test
  • Scenario: Navigating to New Text Page Test
  • Scenario: Deleting Text Item Text
  • Scenario: Data Persistence Test
• New Text Page: NewTextPageFeature
  • Scenario: Navigating to New Text Page Test
  • Scenario Outline: Creating new Text Item Test
  • Scenario: Validating Input Data Test
• View Text Page: ViewTextPageFeature
  • Scenario: Navigating to View Text Page Test
  • Scenario: View Text Details and go back Test

Maintaining a good readability of your Feature Scenarios can definitely benefit you in the long run! 😉

So with that, once you open up your Test Runner, either built in Visual Studio Test Runner or even a 3rd Part Tool like ReSharper Test Runner, all your tests should be discoverable as below…

All the SpecFlow Features are shown with all the Test Scenarios in them, ready to be executed! 😉

Demo Time!

So let’s fire up these UnitTests…

Woah! What a satisfying thing to watch eh! 😀 One of the best things about it is that, we can see each executed step in detail as human-friendly text. SpecFlow FTW!

Github repo:
github.com/UdaraAlwis/XFWithSpecFlow

Once again feel free to grab it on my github! 😉

SpecFlow mindset…

Integrating SpecFlow based Tests in our Xamarin.Form projects brings all the stakeholders together, by promoting Human friendly test language as living documentation.

This is how We do it – Defining these Features and Scenario Steps require a certain mindset, where we need to look at our application or software in an End User’s perspective. You can integrate SpecFlow as a form of Test Driven Development as well, which is what we’re doing at our company, where we extract the Features, Scenarios and their Steps based on the Requirement Specification, and we start building the Application from there on wards.

Perspective of the End User – In the surface it might look quite easy and straightforward to define these test cases step by step, but sometimes it’s not quite the same for a technical person, due to the heavy technicality they’re focused on and they tend to forget the perspective of the end User. As I’ve experienced in my career, adopting this whole Behavior Driven Development paradigm might be confusing for a DEV personnel sometimes. That’s why sometimes its best to get involvement from a QA or BA personnel in the whole preparation process.

Take it easy yo! – In my personal experience, its best to take it easy! Do not over-complicate, over-exaggerate or think in terms of technical aspect. Just think in the simplest of terms, of yourself as an end user, and look at the requirement that needs to be implemented as such. I too had to overcome this problem myself, since I was a long time developer and when I first tried to adopt myself into SpecFlow based testing, it was quite difficult, but eventually I got the hang of it.

Short and simple! – Keep it short and simple, and maintain clear separation between different steps. If you notice a certain step is getting too complicate or longer than usual, then break it down to several steps. The most important thing is to maintain simplicity, for anyone to easily read and understand.

It’s actually Fun! – It might seem like too much work in the beginning, but trust me, once you get the hang of it, its actually fun. It makes total sense to maintain behavior driven development practices with human-readable testing with all stakeholders of the project!

Conclusion!

SpecFlow brings the whole paradigm of Behavior Driven Development for .NET development, where by helping teams bind automation to feature files and share the resulting examples as Living Documentation across the team and stakeholders. You can easily adopt SpecFlow into Xamarin.Forms projects just the same as any other Testing Framework! SpecFlow provides a whole bunch of amazing features for facilitating these great testing capabilities, but I’ve only touched the tip of the ice berg in this article. Please explore more at: https://specflow.org/

Hope this documentation out of my own experience helps any of you fellow Xamarin.Forms devs out there! Share the love! 😀 Cheers yol!

XAMVVM-02 TDD UnitTesting with Xamarin.Forms!

Thinking of how to adopt Testing practices into your Xamarin.Forms project, in this world of test driven software development? 😉 You’re about to find out!

Yes, you can easily incorporate UnitTest cases into your Xamarin.Forms project just like any other .NET solution, and there’s plenty of ways to do with several test frameworks.

As for the sake of clarity let me start off with a bit of basics…

Why UnitTesting?

So that we could write better software, with minimum failure rate. Specially for the continuous development of the software, making sure today’s code logic change doesn’t break any other elements of the application the next day, to simply put.

UnitTesting is a great approach for Test Driven Development based software building, where we test our code from the smallest units we could break into, making sure top to bottom all the bits are executing as expected! 😉

Follow a pattern…

It’s best to keep your UnitTest cases organised and categorised, as it could get messy when your Application grows as well as your test code base. You can organise your test cases according to a certain pattern, either based on your Application Pages or Flows.

Using Page based pattern for organising our test cases, allows us to write the test cases as they originates from each Page context itself. Further more on the base of behavior and functionalities available in each page in the application, as an example Home Page, Shopping Cart Page, Payment Page, and etc…

You could also use a Flow based pattern for your test cases, where you write test cases accordingly each individual feature flow of the application, as an example Login Flow, Order Flow, Payment Flow, and etc…

The choice is up to your requirement and the nature and complexity of your app. Although in my opinion page based approach could be easy to manage the test cases in long run.

Xamarin.Forms and UnitTesting!

Basically we could use any .NET Testing framework to test our Xamarin.Forms projects such as following.

• MsTest
• NUnit
• xUnit, etc…

There are mainly two approaches we could take for UnitTesting our Xamarin.Forms project…

– ViewModel instances testing: This is about creating instances of the ViewModels of the app inside our UnitTest environment and testing each execution flow.

– Runtime Mocked testing: Here we’re creating a mocked app run time using a third party library inside our UnitTest environment and testing each execution flow.

ViewModel based testing, is a very limited form of testing hence there’s no way to perform testing on Navigation and UI Interactions flows. Although this is very easy to implement, it is not very effective, since with Xamarin.Forms applications we’re dealing with user interactions and all. Runtime Mocked testing is actually a much better way, hence we get to go through the actual application UI flow for each our test cases. So let’s do that! 😉

Xamarin.Forms Runtime Mocked UnitTesting!

Like I said above, this is the most effective since it not only saves a lot of development time, but also beneficial for TDD practices for accelerating development.

So we need to mock the Xamarin.Forms runtime without actually launch the app in an emulator or a device, which is going to be done using this awesome library, Xamarin.Forms.Mocks! Allowing us to create an actual instance of the app in memory, mocking the whole Xamarin.Forms run-time, allowing us to navigate through the app as if we’ve actually launched the app.

How to?

First thing we need to create a new UnitTest project in our Xamarin.Forms solution. Then add a reference of our Xamarin.Forms .Net Standard project to it, so that we can instantiate the app and add references from it into our test project. Next we have to add Xamarin.Forms.Mocks, nuget to the test project to emulate the Xamarin.Forms run-time. 😀

Finally, you can straight up create an instance of App class and access your Xamarin.Forms project’s objects in runtime!

But if you’re using a 3rd party MVVM framework like Prism, they usually have their own Container registration, which we need to manually handle ourselves in a mocked run-time environment. On the bright side, it also gives us more flexibility of control, since we could register our own mocked services for Test environment, such as a Location Service! whereas we could return a mocked fake location data when we override the LocationService with our own FakeLocationService. 😉

Here we override the default App class in our Test project with necessary container registrations and store it in a singleton static object, so that we can access it anywhere in our test environment. Oh and alongside we’re initiating our Xamarin.Forms Mocks run-time as well!

Now its all ready to go! Pretty straight forward eh! let’s get down to business.. 😉

Hands on time!

So just to showcase this hands-on awesomeness, I’ve prepared a simple Xamarin.Forms project application, where you can create simple notes, attach location data, attach created date time values and store them locally, allowing you to view them at any time in a list of items. Quite a simple text-pad-like app.

I’ve named this project solution as, XFTextpadApp and it uses Prism as the MVVM framework.

As you can see above is a view of the project structure with the complete MVVM set up. Let me also share a bit of showcasing of the app and its features to give a more in depth context.

Here’s a little peek into features of this demo Xamarin.Forms Textpad app running on iOS!

In the Home page you can view the List of Texts you have saved. By clicking on a Text item in the Home page list you can view that Text item in detail in the View Text page. Then once you click the “New Text” Button it will take you to the New Text page where you can create and save new Texts…

In the New Text page you can create new Texts with a Title and Description of your Text and click “Save” button to save it with the Location and Time data attached to it.

Then you will be navigated back to Home page. In the Home page Text list you can delete saved Text by using the context menu as well…

Thanks to Xamarin.Forms we can deploy it directly to iOS, Android, and UWP as well! 😉

Now the App project is ready, we need to add Test project to it.

Let’s begin by adding the Test project to our Xamarin.Forms project, create a new folder in the solution, named as “Tests” and then right click on it -> Add -> Add New Project

Now this is where the choice is absolutely up to you, as to which testing framework to choose from. Just for this demo, I’m gonna use NUnit Test framework, hence I will select NUnit Test Project (.NET Core) template and create our test project…

I will be naming the test project as, XFTextpadApp.NUnitTests making sure to stick to the standard naming practices! You might want to pay some attention to the Project Path there as well, where as to make sure we set the path to be inside the “Tests” folder that we created earlier.

First thing first, add a reference of the host XFTextpadApp .NET standard project into our XFTextpadApp.NUnitTests project. Then we’re going to add our Xamarin.Forms Mocking library, Xamarin.Forms.Mocks.

Just to make sure we write some beautiful test code, let’s add Shouldly nuget as well. This is more of a personal choice 😉 or if you prefer you could use the default assertions of NUnit (NUnit.Framework.Assert), or which ever the test framework you have chosen to set up.

If all went well then you should see the references as above. Now that’s out of the way, next on let’s get into coding!

Create app instance…

You could directly use the existing App class instance inherited from the Xamarin.Forms project within our Test project environment, but in the case of using 3rd party MVVM frameworks, or even when you want to mock some test compatible services, such as a GPS location service!?, then you need to construct your own App instance that’s catered for the Test environment. Which is what we’re going to do in this step.

Something to note here is that, the way you construct your Test App instance is completely dependent on how you have constructed your Xamarin.Forms application, so try to duplicate it as much as possible.

Let’s start by creating the instance of our app for the test environment. This basically means we need to create App class’s instance with all the container registrations set up, similar to what we already have in our App.xaml.cs file. Let create a new class called TestApp in our XFTextpadApp.NUnitTest project root.

public class TestApp : PrismApplication
{
	public TestApp() : this(null) { }

	public TestApp(IPlatformInitializer initializer) : base(initializer) { }

	protected override async void OnInitialized()
	{
		...
		
		await NavigationService.NavigateAsync("NavigationPage/HomePage");
	}

	protected override void RegisterTypes(IContainerRegistry containerRegistry)
	{
		containerRegistry.RegisterForNavigation<HomePage>();
		...

		containerRegistry.GetContainer().RegisterType<HomePageViewModel>(new Unity.Lifetime.ContainerControlledLifetimeManager());
		...
	}
}

Now make sure to register your Pages, ViewModels and Services accordingly. Also notice that how I have set up the default start up navigation in the OnInitialized() method according to Prism.

Next the Hooks…

This is more like the helper class that we’ll be using to set up the necessary hooks for our test run time environment, such as instantiating the App instance object, and initialising MockForms. Here we’re going to keep a public static object of our App instance, which we will be using all across out tests cases. Let’s create a class called TestHooks…

public class TestHooks
{
	public static TestApp App { get; private set; }

	[SetUp]
	public static void Setup()
	{
		Xamarin.Forms.Mocks.MockForms.Init();

		App = new TestApp();
	}

	[Test]
	public static void IsAppRunning()
	{
		Assert.NotNull(App);
	}
}

We’re using the [Setup] attribute of the NUnit framework to do the initiation of the App instance before each execution of tests. And we’re also adding a simple test in the same class to make sure the instantiation has happened properly without an issue, as you can see by the method IsAppRunning() which is decorated with the [Test] attribute of NUnit. We’ve made those hooking methods static because we are going to be accessing them in our test cases!

And since we have just added our first Test case, we could try and run it, where as if everything is set up properly, you should see your app instantiating well and our first test case is passing! 😉

And it all passes the test with green! lol 😀

If you can’t get this step to pass successfully, then there’s definitely something wrong with the Test App instance set up, so you might wanna look into the DI Container initialization and all.

Alright, but we’re not done yet!

One base for all…

Then we are one step away from creating our Test cases! 😀 so before that we need to create the Base class for our Test classes that would be containing those Test cases. So let’s create out BaseTest class in the root of the project…

/// <summary>
/// Base class for Test classes
/// </summary>
public class BaseTest
{
    /// <summary>
    /// Instance of the app
    /// </summary>
    public static TestApp App { get; private set; }
 
	[SetUp]
	public void Setup()
	{
		TestHooks.Setup();

		App = TestHooks.App;
	}
     
    ...
}

It’s pretty simple, but its important to abstract a base layer for our test classes to dump all the repetitive test steps to reduce duplication of test code. Here we’re simply retaining the TestApp instance through the TestHooks class we just created before, and that instance will be useful for our Test cases later. 😉

On to the final step!

Creating the Tests…

Now that all is set up and ready to go, let’s create our actual Tests. But first create a “Test” folder in the root of our XFTextpadApp.NUnitTest project.

If everything is well set up until now, then it should look like the above structure.

Let’s add a new class to the Tests folder, but as for the name, you could use anything as you wish according to your preferred naming convention, but I would usually prefer something like “<PageName>Tests”, keeping the Page as the objective for the set of test cases. So let me create HomePageTests class, which will inherit from the BaseTest class.

Next we’re going to create the methods that actually indicates the Test cases, let’s take an example such as,

Scenario:
First time I launch the app
I am Navigated to the Home page
I see an empty list of data
I see a label indicating empty data

We could create the Test case for the above scenario as shown below…

[TestFixture]
public class HomePageTests : BaseTest
{
	/// <summary>
	/// Launching the app for the  first
	/// time and navigating in to Home page
	/// </summary>
	[Test]
	public void AppLaunchingFirstTimeTest()
	{
		// Test steps here
	}
}

As you can see I’ve named our test case method as, AppLaunchingFirstTimeTest, considering the gist of the test scenario that we’re targeting. In fact providing a proper name for your test cases are quite important, hence it makes it easier to read and understand, as well as for future maintenance.

Now Thanks to the Xamarin.Forms.Mock, we’re able to pull objects from Xamarin.Forms UI elements and the DI Container to construct a perfect test case for any given scenario. You can access any Xamarin.Forms elements through the App object as usual to check for any values you require.

So with that in mind, let’s write the Test steps for our test case..

[Test]
public void AppLaunchingFirstTimeTest()
{
	// Is the app running
	App.ShouldNotBeNull();
	
	// Retrieve Navigation Stack
	var navigationStack = ((NavigationPage)App.MainPage).Navigation.NavigationStack;
	
	// Am I in the Home page
	navigationStack.Last().BindingContext.GetType().Name.ShouldBe(nameof(HomePageViewModel));

	// ListView should be empty
	App.Container.Resolve<HomePageViewModel>().TextList.Count.ShouldBe(0);

	// Empty ListView Label Displayed
	App.Container.Resolve<HomePageViewModel>().IsEmptyTextList.ShouldBe(true);
}

Let me explain step by step…

First thing we need to do is to make sure the App is running, so in our Test case, let’s access the TestApp instance in our BastTest class, and use Shoudly’s extension to check for null or not state.

Then we access the Navigation stack through the App’s Navigation object, which we’re going to check for the current active Page, through the BindingContext type as you can see.

Finally we check if the ListView is empty by accessing the TextList collection object inside the Homepage ViewModel and we do the same for checking if the Empty list indicator label is visible as well. 😉

There you go, you’ve just created your first UnitTest case for your Xamarin.Forms app!

Mixing it up!

Since our Xamarin.Forms runtime is mocked, you can mix and mingle among the actual UI Elements of your App and DI Container objects.

You see in the previous demo code, how I’ve accessed NavigationStack of the app using App.MainPage instance, as well as you can see how I’m using the Current active Page instance to access its child element, which is a Label element and check its rendered value as follows…

// Get the value in ViewModel
var viewingTextItem = 
   App.Container.Resolve
      <ViewTextPageViewModel>().TextItem;

// Check TextTitle Label value
var textTitleLabel = 
((Label)((Grid)
	((ViewTextPage)GetCurrentPage())
		  .Content).Children.First());
textTitleLabel.Text.ShouldBe(viewingTextItem.TextTitle);

You can see how I have retrieved the Label element by traversing through the Page object, and comparing the results with the value I’ve retrieved from the ViewModel object. So you can mix and match these two approaches in harmony for implementing complicated test cases easily!

App Properties…

Thanks to the mocked Xamarin.Forms runtime we’re able to access most of its functionality as it is in our test cases, which goes to the Application Properties as well.

As an example here I am clearing out my Application Properties on TearDown event after any given test case is executed.

// Clear App properties in memory
Application.Current.Properties.Clear();
await Application.Current.SavePropertiesAsync();

Navigation…

The Navigation in my demo project is handled by Prism, which uses the default Xamarin.Forms navigation underneath it. The Xamrin.Forms.Mock seem to be handling it pretty well as well. So you wouldn’t be need to do any special handles in the test code.

[Test]
public void NavigatingToNewTextPageTest()
{
	...

	// Navigating to New Text page
	App.Container.Resolve<HomePageViewModel>()
		.GoToNewTextPageCommand.Execute();

	// Am I in the New Text page
	GetCurrentPage().BindingContext.GetType()
		.Name.ShouldBe(nameof(NewTextPageViewModel));
}

Hence I could simply call up the usual Navigation Commands that I use on the Button element, here in my unit test code as well. 😉 Navigation will behave as expected!

Mockin’ Services!

Like I mentioned before if you have a Service which relies on platform specific features, then you need to mock those services instances in your testing environment. In my demo I’m using a LocationService, which I have mocked in the my test code as follows…

/// <summary>
/// Mocked Location Service
/// </summary>
public class MockLocationService : ILocationService
{
	public Task<Location> GetLocation()
	{
		// Mock location data
		return Task.FromResult(new Location(0.11111, 0.22222));
	}
}

Also do not forget registration in the TestApp instance which I showed before…

// register mocked ILocationService
containerRegistry.RegisterSingleton
      <ILocationService, MockLocationService>();

Now with that, every time the actual App code try to resolve the LocationService it will get the MockedLocationService object and use its mock data during the test executions. You could also use an awesome library like moq for mocking services and data as well.

Fake em!

If you have actual API calls going in or out of your app, you shouldn’t use them in your unit tests, hence it could cause for slowing down the test executions, and would consume a lot of resources, keeping in mind tests are meant to be executed repeatedly.

So you need to create Mocked Services for those API bound calls, with fake data sets attached to them, so in test code, they would be using the fake data or fake responses instead. 😉 Another reason to use moq library! There are others as well, Faker.net and Bogus are also popular!

Reduce the repetition!

There could be repetitive steps in these Unit Test cases, therefore you can improve the code by extracting out those common repetitive test steps into separate methods or extensions.

public class ViewTextPageTests : BaseTest
{
	[Test]
	public void NavigatingToViewTextPageTest()
	{
		...
		// Create a new Text item
		var textItem = CreateSampleTextItem();
		...
	}

	[Test]
	public void ViewTextDetailsAndGoBackPageTest()
	{
		...
		// Create a new Text item
		var textItem = CreateSampleTextItem();
		...
	}

	public TextItem CreateSampleTextItem()
	{ ... }
}

As an example in both my test cases above, I need to create a new TextItem object, hence I’ve extracted it out to a common method which is used in both, reducing the repetition.

Here’s another example where I’ve created the following helpers in BaseTest class,

/// <summary>
/// Returns the active Page
/// </summary>
/// <returns></returns>
public static Page GetCurrentPage()
{
	var navigationStack = ((NavigationPage)App.MainPage).Navigation.NavigationStack;
	return (Page)navigationStack.Last();
}

/// <summary>
/// Returns the active ViewModel
/// </summary>
/// <returns></returns>
public static ViewModelBase GetCurrentViewModel()
{
	var navigationStack = ((NavigationPage)App.MainPage).Navigation.NavigationStack;
	return (ViewModelBase)navigationStack.Last().BindingContext;
}

Allowing me to retrieve the current active Page or the ViewModel directly.

Before and After Test Case…

Most Testing frameworks provides you with Before and After hooks for a given Test Case, where as in my demo, NUnit provides [Setup] and [TearDown] subsequently.

public class BaseTest
{
	[SetUp]
	public void Setup()
	{
		// Set up steps
	}
	
	[TearDown]
	public async Task TearDown()
	{
		// Teardown steps
	}
     
    ...
}

As I’ve shared before I’ve set up the [Setup] attribute in the BaseTest class, allowing me to reinstantiate TestApp instance before each Test Case executes.

[TestFixture]
public class HomePageTests : BaseTest
{
	[SetUp]
	public new void Setup()
	{
		// Page specific Setup
	}

	...
}

You can override the set up in your Page specific test class and set up the required customized setting up as well. This was you can also share data across the whole flow of the page test scope without resetting the TestApp instance. Quite a handy trick to keep in mind! 😉

Finalizing…

So all of this awesomeness is committed into my github repo:

Github repo:
github.com/UdaraAlwis/XFWithUnitTest

Since I took the Page based pattern for organising my unit test cases, here’s the set of test classes I ended up creating for each page.

Under the Test folder I’ve placed all my test class, but you can restructure this hierarchy as you please, even have sub folders containing different sets of test as well… 😉

So in my demo below are the test classes that represents each page…

• Home Page : HomePageTests
• New Text Page: NewTextPageTests
• View Text Page: ViewTextPageTests

Maintaining a good readability of your test cases can definitely benefit you in the long run! 😉

Once you open up your Test Runner, either built in Visual Studio Test Runner or even a 3rd Part Tool like Resharper Test Runner, all your tests should be discoverable as below…

All the UnitTest classes are shown with all the test cases in them, ready to be executed! 😉

Demo time!

So let’s fire up these UnitTests…

Woah! What a satisfying thing to watch eh! 😀

Github repo:
github.com/UdaraAlwis/XFWithUnitTest

Once again feel free to grab it on my github! 😉

Conclusion

You can easily implement UnitTesting in Xamarin.Forms just like any other .NET project, and there are plenty of ways to do it. Mocking Xamarin.Forms runtime and implementing our UnitTests on top of that, is a great way to ensure that our tests are executing as close as it gets to the real app runtime. You can easily integrate Xamarin.Forms.Mocks library with pure Xamarin.Forms or along side any other MVVM Library to set up your UnitTests!

We need to make sure to follow best practices and maintain clean, easy to read, test code across all our UnitTesting implementations. You can take different approaches to organising your test cases, while I personally believe Page-Context based pattern provides more clarity.

 Hope this documentation out of my own experience helps any of you fellow Xamarin.Forms devs out there! Share the love! 😀 Cheers yol!

Prepping your Xamarin.Forms App for Penetration Testing!

Let’s make sure your Xamarin.Forms App is ready for Penetration Testing, or widely known as PEN Testing! Focused on the Network Communication Layer, so that we can have a peace of mind for ourselves without being annoyed by our QA and Security Analysis team! 😉 lol

In any Mobile App Development project, we need to thoroughly test our Mobile Apps for vulnerabilities and security of user data. That is why these penetration testing processes are very important, we need to make sure that we deliver a mobile application as hardened and safe guarded as possible for our Users.

Now this post is not about securing or hardening the security of your mobile app, but rather how to prepare your Xamarin.Forms built app for QA and PEN Testing procedures.

But why, Xamarin.Forms?

Xamarin.Forms does an incredible job at producing a almost native-like level Mobile Application at the end of development. But given the unique nature of Xamarin.Forms, the .NET framework that we work with, sometimes developers take it for granted, giving all the responsibility to the framework, and we miss some bits that we need to pay attention to in oppose to native mobile development.

We need to make sure our App is compatible with the QA and PEN Testing procedures. Most of the QA and PEN testing revolves around the Native Mobile App environment. But sometimes those typical native mobile app testing processes aren’t compatible with what we work on out of the box of default .NET builds of Xamarin.Forms!

PEN Testing of a Mobile App…

Now there are many different kinds of QA procedures and PEN Test cases, such as local DB analysis, MiTM packet analysis, etc. In order to support those different PEN Test procedures, some times we need to provide separate builds with different configurations of your app to our PEN Test team. Such as disabling SSL Pinning in the app so that they can execute MiTM packet analysis testing on our app, and there could be many different scenarios as such.

So I’m going to share with you some of the important key points that you need to make sure you have configured in your Xamarin.Forms project, specifically in your Network Communication Layer and for some of those custom builds that you might have to provide for the PEN Testing process.

HttpClient Handler setup..

While Xamarin does provide a full fledged Managed HttpClient and Handler, it is best to set up the HattpClient’s handlers to the device native handlers. This will make sure better performance and better native system level security for your app.

On your Xamarin.Forms project solution, go to the Android project node -> Properties -> Android Options -> Click “Advanced” button and take a look at HttpClient implementation and SSL/TLS implementation.

Make sure to set them up as above using the native Android handler, and Default (Native TLS 1.2+) Transport Layer Security for all the web calls. This will make sure all our web endpoint calls will be handled by those configurations which are best suited for performance and security of the native android system.

Then for the iOS, On your Xamarin.Forms project solution, go to the iOS project node -> Properties -> iOS Build and take a look at HttpClient implementation.

Make sure its set to NSUrlSession handler, which will provide your iOS app with native iOS level security and better performance for web endpoint calls. Also this means your app won’t support devices before iOS version 7, so better check your app requirements as well.

HttpClient setup in Xamarin.Forms!

It is very crucial that you imperilment the use of HttpClient in your Xamarin.Forms app properly with performance and security in mind. It would be best to register the instance of HttpClient as a Singleton, and refer to that singular instance for all your web endpoint calls. This will not only make it easy for debugging, also easy for configuring your API/Web endpoint execution layer’s implementation.

And set up a HttpClientHandler to be passed into your HttpClient instance during the instantiation, so that we can include all the custom configuration easily.

Unless its a must, make sure to use the Native HttpClientHandlers for your HttpClient, which will increase the performance and native security features for your app.

• Android: AndroidClientHandler
• iOS: NSUrlSessionHandler
• UWP: WinHttpHandler

Instead using the .NET Managed HttpClientHandler, use the above instances mapped up to your Xamarin.Forms shared environment. Here’s something that might be useful: Xamarin and the HttpClient For iOS, Android and Windows

Do not re-instantiate the same HttpClientHandler nor HttpClient instance during run time and keep those different configuration separately for each HttpClient by registering multiple types with a pre-defined use case. Such as AuthClient for handling authentication and ApiClient for normal endpoint calls.

Although there are third party HttpClient libraries such as ModernHttpClient, that provide better features for these specific scenarios, so you could even try one of them! 😉

Disable SSL Certificate Validation..

In case if you needed to disable HTTPS / SSL Certificate validation for your PEN Test procedures such as Man-in-The-Middle packet trace analysis, then we can easily disable this by overriding the Certificate validation execution in the HttpClientHandler and assigning that to HttpClient.

var handler = new HttpClientHandler();
handler.ServerCertificateCustomValidationCallback =
	(message, certificate, chain, sslPolicyErrors) => true;

HttpClient = new HttpClient(handler);

 

The above applies of course if you have set up Managed HttpClientHandler in your Xamarin.Android and Xamarin.iOS settings instead of using the native handlers.

If you have set up your projects with Native HttpClientHandler then you can easily disable HTTPS Certificate validation by following ways.

For Android, create a Custom derived implementation of AndroidClientHandler as follows,

public class CustomAndroidClientHandler : AndroidClientHandler
{
	protected override async Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
	{
		request.Version = new System.Version(2, 0);
		return await base.SendAsync(request, cancellationToken);
	}

	protected override SSLSocketFactory ConfigureCustomSSLSocketFactory(HttpsURLConnection connection)
	{
		return SSLCertificateSocketFactory.GetInsecure(0, null);
	}

	protected override IHostnameVerifier GetSSLHostnameVerifier(HttpsURLConnection connection)
	{
		return new BypassHostnameVerifier();
	}
}

internal class BypassHostnameVerifier : Java.Lang.Object, IHostnameVerifier
{
	public bool Verify(string hostname, ISSLSession session)
	{
		return true;
	}
}

Based on: https://nicksnettravels.builttoroam.com/android-certificates/

for iOS, subscribe to TrustOverride and return true to override certificate validation.

var iosClientHandler = new NSUrlSessionHandler();
iosClientHandler.TrustOverride += (sender, trust) =>
{
	return true;
};

 

For UWP: subscribe to ServerCertificateValidationCallback and return true to override certificate validation.

var uwpClientHandler = new WinHttpHandler() 
{ 
	ServerCertificateValidationCallback = 
		(message, certificate2, arg3, arg4) =>
		{
			return true;
		}
};

 

As you can see in all the snippets above we’re overriding HTTPS Certificate Validation process manually. Now this is not a build to be pushed for Production, make sure to produce this build as a Test only build for PEN testing. So I would suggest keep this configuration in a separate branch build and opt back to the main branch for production release.

Enable Self-Signed SSL Certificates..

Instead of disabling entire SSL Certificate validation process, we could narrow down the override to a certain SSL Certificates, such as a Self-Signed Certificate endpoints that our PEN Testers might be using.

Although there are many ways to do this, basically we include the given self-signed certificate data in the app’s configuration, or override the validation with it’s key. I would recommend going into the following tutorials for it hence it covers a wide aspect of it for both iOS and Android.

Self Signed iOS Certifcates and Certificate Pinning in a Xamarin.Forms application

Self Signed Android Certificates and Certificate Pinning in Xamarin.Forms

So kudos to nicksnettravels blog! 😉 They’ve got great stuff in there!

Enable non-HTTPS!

As you know by default Android (since Android P) and iOS platforms doesn’t allow insecure non-HTTPS calls to be made from our apps,

• Android P: Protecting users with TLS by default in Android P
• iOS: Preventing Insecure Network Connections

Now we might have to disable this during development until the back end server is TSL enabled or even for a PEN test case with a custom build.

Let’s disable this on Android by using android:networkSecurityConfig in your AndroidManifest.xml with a reference to the @xml/network_security_config which we’ll create next.

<manifest ... >
    <application
        android:networkSecurityConfig="@xml/network_security_config"
        ... >
        <!-- Place child elements of <application> element here. -->
    </application>
</manifest>

 

Add a new folder called “xml” in your Resources folder and add a new xml file with the name network_security_config.xml this will hold the key to disable or enable HTTPS connections restriction as you wish.

<?xml version="1.0" encoding="utf-8" ?>
<network-security-config>
  <domain-config cleartextTrafficPermitted="true" />
</network-security-config>

 

You can set cleartextTrafficPermitted to false later during production build.

And on iOS add the following configuration in the info.plist file.

<key>NSAppTransportSecurity</key>
<dict>
    <key>NSAllowsArbitraryLoads</key>
    <true/>
</dict>

 

Oh once again, make sure to disable this during PROD builds!

System Proxy Settings support!

Something to keep in mind is that you don’t need to worry about System Proxy settings as long as you’re using the Native HttpClientHandlers as they would by default use the system preferences under the hood, but if you’re using Managed .NET HttpClientHandlers then you need to handle the System native Proxy Settings.

You can do this by creating a Platform specific Service that will extract the System Proxy Settings, and you can use that when you’re instantiating HttpClientHanlder. This article provides a great example for it: HttpClient and Proxy

Is all good? 😉

Now before you ship out your build for the PEN test you can make sure everything is in place according to your specific requirements.

BadSSL (https://badssl.com/) is a great web tool to check whether your app is able to access non-https endpoints or self-signed certificate endpoints and so on based on your PEN Test requirement. You can call upon those test endpoints directly from your app to make sure they’re accessible or not.

Network monitoring tools like Fiddler and Charles Proxy, allows you to set up dummy local proxy servers that you can test with in order to make sure your app supports proxy communication or in that case App’s compatibility with using System proxy settings. You could even monitor the data streams going in and out of the app, and see if they’re properly secured with encryption.

So for this demo I prepared a little sample app which showcases the use of non-HTTPS, Self-Signed endpoints access, and Proxy settings reading capabilities. Check out below Android, iOS and UWP run times…

As you can see it’s able to go through all the defined scenarios I mentioned before. For this demo app I’m using all Native HttpClientHandlers, so I’m using the exact code snippets I’ve shared in this blog post. Now let me try to emulate some failure scenarios where its missing some configuration I shared in this post.

Full demo code: github.com/UdaraAlwis/XFPenTestPrepDemo

   

1st I’m emulating the instance where Self-Signed HTTPS endpoints unauthorized to execute as well as Non-HTTPS endpoints blocked by default System Security layer. 2nd one shows where the Self-Signed HTTPS is diabled but non-HTTPS endpoints allowed and so on to the 3rd scenario.

Well that’s it. Hope this helps you to configure your app properly with security in mind and provide builds for the PEN Test processes according to the requirements. 🙂

Share the love! Cheers! 😀

XAMVVM-01 A real walk-through of Xamarin.UITest with Xamarin.Forms!

Let’s take a chilled out walk through adding Xamarin.UITests to your Xamarin.Forms project! 😉

When I said chilled out, I meant literally a chill, no-fuss walk through adding UITests for your Xamarin.Forms project solution. There’s many articles and tutorials out there regarding this, but when I first started I couldn’t find a clear enough guide to begin with for myself, so I was stumbling around in confusion here and there until I self-learned a proper foundation. That’s why I thought of writing this post.

This is an attempt of sharing my experience and intuition of how to architect a better UITest structure for your Xamarin.Forms Solution helping you to get a clear and easy head start!

So I’m not gonna get into extreme baby steps, or details, but a clear fuss-free hands-on guide for starting off with your UITests, which I hope would give you a clear picture and understand of the whole shabang! After all I’m all about that solid project architecture!

Why UITests?

Hey, if you’re looking for a serious intro, please Google! I don’t like repetition of content lol.

Xamarin.UITests comes in handy when you want to have consistent assurance of the actual functionality of the app with the UI behaviour included. And between you and me, I actually love watching UITests being executed on Devices and Simulators, seeing you app actually being used like a human, giving you a whole feedback loop of the UI behaviour , is just an incredible experience! 😉

 

Let’s get started..

Just for the showcase of this awesomeness, I created a little App, which is called Textpad, where you simple take notes or texts of whatever you feel like. 😉 A very simple out of the box Xamarin.Forms app, and fully MVVM architectured code base with Prism. I named the solution as “XFWithUITest” just for this demo.

Whatever the default template of the Xamarin.UITest has provided, I have done several changes to it here and there for the clarity and of the code base as you will see in this article.

So I’m gonna walk you through a clean and well-structured manner of adding Xamarin.UITests to your project solution.

You can take a little sneak peak at it over here in my github repo:
XAMVVM-Playground/XFWithUITest

Structure is important!

There’s many ways to structure a UITest, but I like a clean separation of the elements in any solution architecture. Like here we’re going to separate our Tests from the actual Host projects.

So first, for the name of separation let’s add a Folder called “Tests” in your Xamarin.Forms solution. Yes, that’s the way to start!

Then let’s create our Xamarin.UITest project, right-click on the “Tests” folder in the VS Solution Explorer and go to Test tab and select Xamarin.UITest Cross-Platform Test Project!

Also pay extra attention to the Name and Location value for our UITest project. Append “.UITest” at the end of your project name. As of the location, make sure to add the path along with the “Tests” folder that we created above.

Next create a new Folder inside of that project called “Tests”, yes another one, which is where we’re actually placing our tests! Also create a new class called SetupHooks, which is where we’ll maintain all the hooks that are needed for our tests. (I’ll get into details for this in a later step)

Now it should look something like this!

Nothing more.

Delete anything else that’s unnecessary or not seen above! 😉

Off to next step!

Don’t forget the nugets!

Make sure all the necessary nuget packages are in place, which is just basically the following 3 nugets! yep that’s it!

Pay very careful attention here to the version of NUnit version 2.6.4, which is the minimum NUnit version supported by Xamarin.UITest as of today. (01/11/2018)

The deal with AppInitializer!

Now this right here is where your Tests will be firing up the app’s execution. There are many ways to structure this class and its functionality, but here’s my way…

This class comes pre-populated when you first create the UITest project, but I have made some changes of my own for the clarity of the code.

As you can see I’m passing in an extra boolean parameter “clearData”, which is to execute a clean instance of my App for testing.

I’m using the InstalledApp() call to load the Android and the iOS apps from the simulators, also I’m enabling the EnableLocalScreenshots() to get actual screenshots of my test instances as I wish. Yeah the fact that you can automatically capture screenshots during testing even when you run locally is really cool feature of Xamarin.UITests! 😉

Now instead of getting a hook on the InstalledApp(), you could use the path to the APK or IPA file using the ApkPath() or AppBundle() respective for Android and iOS, which is totally up to your choice.

Then I’m passing in the AppDataMode parameter according to my choosing of the “clearData” value.

SetupHooks holds the instances!

Remember earlier I created a class called SetupHooks? let’s set it up now!

public class SetupHooks
{
      public static IApp App { get; set; }

      public static Platform Platform { get; set; }
}

 

During UITests execution we’re holding a singular instance of the app in memory, which we’re calling through UITest’s functions to perform many operations, so to simplify that, here we’re holding a public static instance of the IApp and Platform object to be used in our Test cases.

Pretty neat eh! 😀

Let’s write the Tests!

Create a class called AppTests, which is where we’re going to place the Test fire up code and the rest of the tests for now!

namespace XFWithUITest.UITest.Tests
{
    [TestFixture(Platform.Android)]
    //[TestFixture(Platform.iOS)]
    public class AppTests
    { 
        public AppTests(Platform platform)
        {
            SetupHooks.Platform = platform;
        }

        [SetUp]
        public void BeforeEachTest()
        {
            SetupHooks.App =  
            AppInitializer.StartApp(SetupHooks.Platform, true);
        }

        [Test]
        ...
	// test cases begin here...
		
	}
}

 

There I have added the TestFixture attributes as required by NUnit to identify our tests, and notice how I have commented out the iOS platform, to show you that you could stick to one platform at a time for your ease of testing, instead of seeing failed tests in the Test Runner window! 😉

[SetUp] is where your Tests will initialize the actual App instance, thus retrieving a hook to the app’s instance for our Test cases to use.

You can see how I’m tying up the SetupHooks – Platform and App instances, through the initiation of the AppTests.

AppInitializer.StartApp(SetupHooks.Platform, true);

This gives a clean instance of the app for our tests cases to use, and up on your wish you could pass in “false” to the same method and get a data persisted instance of the app at anytime, anywhere in your tests! 😉

Now you’re all set to start writing your UITests, but before we begin I need you to check up on something else!

AutomationId for everything!

Whatever the UI element you need to get a hook on to or get a reference of, be it a Page, Button, Layout even a Label, you need to add a value to its AutomationId.

And make sure every AutomationId in a given Page context is unique for every element, otherwise the look up function will return all the elements that matches the given Id, which could lead to confusion in your tests 😉

IApp interface functions!

The Xamarin.UITest.IApp interface provides a whole bunch of functionalities for the app for us to play around with in order to execute our test scenarios.

Take a look here, Xamarin.UITest.IApp to see the list of powerful functions we can use. To name a few are Tap, Swipe, Scroll, WaitForElement and etc, to be performed on any given UI Element on the screen.

So now all you need to do is get a hook on any given element..

Getting a hook on an Element…

There’s several ways of doing this, most common is by the AutomationId of the Element

SetupHooks.App.Tap(c => c.Marked("Button1"))

Another is by the value of an Element’s property,

SetupHooks.App.Tap(c => c.Text("Click this Button!"))

Or you could do by even the Class name of the element. Choice is completely yours, pick the one best suited for your test case.

How to write a Test?

Now this is the coolest part, Xamarin.UITest allows us to get hooks on to UI Elements of the running App, then we perform actions on those elements and wait for the results and check if it resulted as expected through assertion using NUnit.

So its basically a little dance between Xamarin.UITest and NUnit Assertion! 😉

As a standard keep in mind to append “Test” at the end of each of your Test cases.

As you can see above I’m first waiting for the HomePage to appear, then I’m asserting it through NUnit. Then I look for the Label with “Hey there, Welcome!” text!

Action and Result, simple as that! 😀

Some advanced bits…

Here’s some advanced bits that could come in handy!

Getting the number of elements in a ListView

SetupHooks.App.Query(c => c.Marked("TextListView").Child()).Length

Getting an element in a ListView

Func<AppQuery, AppQuery> itemInListView = null;

if (SetupHooks.Platform == Platform.Android)
     itemInListView = 
     x => x.Class("ViewCellRenderer_ViewCellContainer").Index(0);
else if (SetupHooks.Platform == Platform.iOS)
     itemInListView = 
     x => x.Marked("<your listview automationId>").Index(0);

// change the index parameter to get the item you wish

Opening Context Menu in a ListView item

// pop up the Context menu in ListView item
if (SetupHooks.Platform == Platform.Android)
      SetupHooks.App.TouchAndHold(firstCellInListView);
else if (SetupHooks.Platform == Platform.iOS)
      SetupHooks.App.SwipeRightToLeft(firstCellInListView);

Enter Text into an Entry or Editor

SetupHooks.App.EnterText(
c => c.Marked("TextTitleEditor"), whateverYourText);

Wait for an element to disappear

SetupHooks.App.WaitForNoElement(c => c.Text("This label text"));

// either by Text or Marked as should work

Restarting the app anywhere…

// restarting app, persisting state

SetupHooks.App = AppInitializer.StartApp(SetupHooks.Platform, false);

Check out more here in this awesome git page: XamarinTestCloudReference

REPL is your tool!

Yes start using the REPL command line to see how your App’s UI is actually rendered by the native platform at any given execution time. Simply call this anywhere you wish in the UITests steps,

App.REPL();

And you’ll be presented with a CLI which will help you see the whole UI tree of the screen. Simply type “tree” in the CLI and you’re good!

Structuring the tests..

Now there’s many ways to structure all the test cases and scenarios, and there’s no strict standard way that should be followed, but whatever you’re comfortable or fits your project is totally fine and the choice is yours!

You could include all your Test cases in the AppTest class itself, or you can break them into separate classes regarding the Page, or the functionality type.

So for this demo I’m keeping all my UITest cases in the AppTest class itself.

Running the UITests locally!

Well now that we have structured the architecture, here’s the time for actual firing things up and you’ve got couple of things to remember!

You can run your Android Tests on Simulator and Device directly without any modification as long as you provide the right APK path or the App Id.

You can run your iOS Tests only on Visual Studio for Mac, and for the device you need to pass the provisioning details, and as of simulator, you need to pass the Simulator Id.

If you’re using InstalledApp() or ConnectToApp() in your AppInitializer, then make sure the app is already deployed or running in the devices or simulator.

Also make sure to keep your Devices or Simulators or Emulators screens switched on at all times, otherwise tests will break giving a waiting exception.

That’s it!

But I’m not completely satisfied with the architecture, so let’s kick it up a notch! 😀

Little cherry on top Architecture!

Like I said before there’s many ways to construct the architecture for your Test project, one of my favourite ways is by separating the test cases by Page Scenario, which I think is a much cleaner structure.

We’re going to create a base class, “TestBase” which has the constructor initiation and BeforeEachTest setup, then create a sub classes that inherits from it representing whatever the pages we have in the App.

It should look something like this!

And don’t forget you need to add TestFixture attribute for every single sub-class!

So what you’re gonna do is take apart all the Test cases you had in one class and move them into the related pages, simply cut and paste of the methods should do! Also on top of that you could abstract another layer of shared steps that we could reuse across these Page tests. 😀

Then it should give you a clean Test output as below.

There you go, all the Tests are now nicely aligned and structured under the given Page which it associates with!

Pretty neat eh!

So this above structure of mine is somewhat more of a simplification of the Page Object Architecture which is well explained here for Xamarin.UITests: https://www.codetraveler.io/

And even in this official github sample from Xamarin uses the same similar pattern: SmartHotel.Clients.UITests

Done!

As you can see its not that hard to set up your Xamarin.Forms project with UITest once you get the basic understanding of the moving parts and keep a clear structure in your head.

Now for some of you might be experiencing some issues with Xamarin.UITest, in which case I had too when I was first starting off. Therefore I ended up writing this post sharing my experience of solving them: Getting your Xamarin UITests to actually work! So if you’re having any issues getting your Xamarin.UITests to work in Visual Studio, that post might be able to help you. 🙂

Do check out my Github repo of this post:
XAMVVM-Playground/XFWithUITest

Thus concludes my real walk-through of Xamarin.UITests with Xamarin.Forms, in which I hope you got a clear understanding of how to properly structure your project and all the moving bits and pieces that gets the job done! 😀

Share the love! 😀

Cheers!

Getting your Xamarin UITests to actually work! (Not thanks to Xamarin Docs)

Can’t get your Xamarin UITest to work? then this might actually help! 😉 yeah no thank you Xamarin Documentation!

Backstory?

So there I was having my Visual Studio updated to the latest version 15.8.8 and I created a fresh Xamarin.Forms project using the default template. Then I happily added the Xamarin.UITest project as well me being a TDD enthusiast.

So I wrote some code in my Xamarin.Forms project, added some pages, some simple navigation and functionality. Then I tried to run my Xamarin UITest, and surprise! I couldn’t get it to work at all.

First issue..

Everything was building nice and well, but when I tried to run the UITest, Visual Studio was annoying me with the following error in Output log.

Exception thrown: 'System.Exception' in Xamarin.UITest.dll
An exception of type 'System.Exception' occurred in Xamarin.UITest.dll but was not handled in user code
The running adb server is incompatible with the Android SDK version in use by UITest:
C:\Program Files (x86)\Android\android-sdk

You probably have multiple installations of the Android SDK and should update them or ensure that your IDE, simulator and shell all use the same instance. The ANDROID_HOME environment variable can effect this.

So I tried Debugging the UITest, and no surprise the runtime error was popping up same as above.

In utter surprise I went ahead to the Xamarin UITest documentation to see if there’s anything new that I haven’t heard before which needs to be done to get this super simple test project to run.

But nope! I had done everything as it was mentioned in the documentation with the perfect configuration in my up-to date Visual Studio and project set up already.

So what could have simply gone wrong?

After many days of researching, testing and pulling out my hair, I figured out the problem.

Xamarin.UITest version 2.2.6 (the latest to date: 26/10/2018)) does not support the latest version Android SDK Platform-Tools!

Yeah, no thank you to the Xamarin Documentation that never mentions there’s a constraint on the Xamarin.UITest supported Android SDK Platform-Tools!

Xamarin.UITest version 2.2.6 supports Android SDK Platform-Tools version 25.0.3 or lower only! (to the date: 26/10/2018))

I was actually surprised that there was no mentioning of this anywhere in the Xamarin UITest documentation! Sincerely hope they add some documentation between the support versions of Xamarin.UITest and Android SDK Platform-Tools.

Solving it!

So I went ahead and downgraded my  Android SDK Platform-Tools to version 25.0.3in my PC.

And finally got my UITest to run like a charm! 😀

Don’t you just hate it when an overhyped framework just doesn’t work as expected even after you perfectly set up everything accordingly to their documentation. 😦

Second issue..

Not only that, if you had followed the Xamarin UITest documentation and created your Xamarin.UITest project, then you may be facing the bellow error in your Error List window in Visual Studio!

This is occurring when you add the Android project reference to your Xamarin.UITest project. This actually doesn’t cause any issues on the compilation or runtime, the errors just remain there without going away.

I did a bit of digging on this issue and found out that this is a bug in Visual Studio, which has apparently been fixed in one of the previous versions in VS as Microsoft claims but its still occurring in my dev environment which has the latest version of VS 15.8.8!

Check it out here: https://github.com/MicrosoftDocs/xamarin-docs/issues/508

Since it doesn’t actually interfere in compilation or runtime, it shouldn’t really matter, but it would prevent you from adding Nuget packages to your Xamain.UITest project.

Solving it!

Simple forget what the documentation says and remove the Android project reference from your Xamain.UITest project.

There you got rid of the annoyance! 😉

Still having issues? Then these..

So if you’re still facing any issues pay attention to the following facts and give it a go!

1. Configure ANDROID_HOME environment variable in your Windows system.

Make sure you have added the User variable for ANDROID_HOME path in your Environment Variables.

2. Install the Xamarin.UITest compatible NUnit version 2.6.4 nuget in your UITest project

Make sure you have the exact version as shown above which is the NUnit version supported by Xamarin.UITest as of today.  And you’re gonna need NUnitTestAdapter verion 2.1.1 for the tests to work on Windows.

3. Make sure your AppInitializer is properly configured with apk path, or apk name.

public class AppInitializer
{
    public static IApp StartApp(Platform platform)
    {
        if (platform == Platform.Android)
        {
            return ConfigureApp.Android
                .InstalledApp("com.udara.xfwithuitest")
                .StartApp(AppDataMode.Clear);
        }

        return ConfigureApp.iOS
                .InstalledApp("com.udara.xfwithuitest")
                .StartApp(AppDataMode.Clear);
    }
}

 

There are several ways to configure your APK or IPA file, either with the name or the full file path. Make sure you have chosen best way possible for you. Above is how I have configured mine, which depends on the already installed APK or the IPA file in the device.

4. Don’t forget to configure Calabash to run the UITest for iOS

This is essential if you’re trying to get it to run on iOS, just in case if you had missed this, this is also mentioned in the documentation.

Hope that helped!

Here are some good resources if you’re getting started:

• https://www.jimbobbennett.io/ui-testing-your-xamarin-apps/
• https://robgibbens.com/bdd-tests-with-xamarin-uitest-and-specflow/

Spread the love! Cheers! 😀