Getting your Xamarin UITests to actually work! (Not thanks to Xamarin Docs)

Can’t get your Xamarin UITest to work? then this might actually help! 😉 yeah no thank you Xamarin Documentation!

Backstory?

So there I was having my Visual Studio updated to the latest version 15.8.8 and I created a fresh Xamarin.Forms project using the default template. Then I happily added the Xamarin.UITest project as well me being a TDD enthusiast.

So I wrote some code in my Xamarin.Forms project, added some pages, some simple navigation and functionality. Then I tried to run my Xamarin UITest, and surprise! I couldn’t get it to work at all.

First issue..

Everything was building nice and well, but when I tried to run the UITest, Visual Studio was annoying me with the following error in Output log.

Exception thrown: 'System.Exception' in Xamarin.UITest.dll
An exception of type 'System.Exception' occurred in Xamarin.UITest.dll but was not handled in user code
The running adb server is incompatible with the Android SDK version in use by UITest:
C:\Program Files (x86)\Android\android-sdk

You probably have multiple installations of the Android SDK and should update them or ensure that your IDE, simulator and shell all use the same instance. The ANDROID_HOME environment variable can effect this.

So I tried Debugging the UITest, and no surprise the runtime error was popping up same as above.

In utter surprise I went ahead to the Xamarin UITest documentation to see if there’s anything new that I haven’t heard before which needs to be done to get this super simple test project to run.

But nope! I had done everything as it was mentioned in the documentation with the perfect configuration in my up-to date Visual Studio and project set up already.

So what could have simply gone wrong?

After many days of researching, testing and pulling out my hair, I figured out the problem.

Xamarin.UITest version 2.2.6 (the latest to date: 26/10/2018)) does not support the latest version Android SDK Platform-Tools!

Yeah, no thank you to the Xamarin Documentation that never mentions there’s a constraint on the Xamarin.UITest supported Android SDK Platform-Tools!

Xamarin.UITest version 2.2.6 supports Android SDK Platform-Tools version 25.0.3 or lower only! (to the date: 26/10/2018))

I was actually surprised that there was no mentioning of this anywhere in the Xamarin UITest documentation! Sincerely hope they add some documentation between the support versions of Xamarin.UITest and Android SDK Platform-Tools.

Solving it!

So I went ahead and downgraded my  Android SDK Platform-Tools to version 25.0.3in my PC.

And finally got my UITest to run like a charm! 😀

Don’t you just hate it when an overhyped framework just doesn’t work as expected even after you perfectly set up everything accordingly to their documentation. 😦

Second issue..

Not only that, if you had followed the Xamarin UITest documentation and created your Xamarin.UITest project, then you may be facing the bellow error in your Error List window in Visual Studio!

This is occurring when you add the Android project reference to your Xamarin.UITest project. This actually doesn’t cause any issues on the compilation or runtime, the errors just remain there without going away.

I did a bit of digging on this issue and found out that this is a bug in Visual Studio, which has apparently been fixed in one of the previous versions in VS as Microsoft claims but its still occurring in my dev environment which has the latest version of VS 15.8.8!

Check it out here: https://github.com/MicrosoftDocs/xamarin-docs/issues/508

Since it doesn’t actually interfere in compilation or runtime, it shouldn’t really matter, but it would prevent you from adding Nuget packages to your Xamain.UITest project.

Solving it!

Simple forget what the documentation says and remove the Android project reference from your Xamain.UITest project.

There you got rid of the annoyance! 😉

Still having issues? Then these..

So if you’re still facing any issues pay attention to the following facts and give it a go!

1. Configure ANDROID_HOME environment variable in your Windows system.

Make sure you have added the User variable for ANDROID_HOME path in your Environment Variables.

2. Install the Xamarin.UITest compatible NUnit version 2.6.4 nuget in your UITest project

Make sure you have the exact version as shown above which is the NUnit version supported by Xamarin.UITest as of today.  And you’re gonna need NUnitTestAdapter verion 2.1.1 for the tests to work on Windows.

3. Make sure your AppInitializer is properly configured with apk path, or apk name.

public class AppInitializer
{
    public static IApp StartApp(Platform platform)
    {
        if (platform == Platform.Android)
        {
            return ConfigureApp.Android
                .InstalledApp("com.udara.xfwithuitest")
                .StartApp(AppDataMode.Clear);
        }

        return ConfigureApp.iOS
                .InstalledApp("com.udara.xfwithuitest")
                .StartApp(AppDataMode.Clear);
    }
}

 

There are several ways to configure your APK or IPA file, either with the name or the full file path. Make sure you have chosen best way possible for you. Above is how I have configured mine, which depends on the already installed APK or the IPA file in the device.

4. Don’t forget to configure Calabash to run the UITest for iOS

This is essential if you’re trying to get it to run on iOS, just in case if you had missed this, this is also mentioned in the documentation.

Hope that helped!

Here are some good resources if you’re getting started:

• https://www.jimbobbennett.io/ui-testing-your-xamarin-apps/
• https://robgibbens.com/bdd-tests-with-xamarin-uitest-and-specflow/

Spread the love! Cheers! 😀