Category Archives: Code

MS Network 6: Neum, Bosnia and Herzegovina

Thanks to everyone who came to my talks at MS network 6 conference.

Here are the slides and demo files from my presentations:

Building an Adaptive UI for UWP apps        Slides     Demos

Data in UWP Apps: XML, JSON, SQLite, Entity Framework Core    Slides     Demos

Remember: to run the demos using SQLite, you must first go to Visual Studio ToolsUpdates and Extensions, select the Online category and search for SQLite for Universal Windows Apps. Install that package.

You must also go into the Project References for the projects, and if the SQLite for Universal Windows App Platform is listed but there is a yellow bang icon over it, delete the reference and then re-add it through Add Reference – you will find SQLite for Universal Windows Platform listed in the Extensions list under Universal Windows.

Once again – thanks for hosting me in Bosnia and Herzogovina Smile

Working with SQLite in UWP

*Post will be updated with explanation soon!*

The following samples have been tested on Desktop, Mobile and IoT (Raspberry Pi 2). They require that you have the 10586 SDK installed (Visual Studio 2015 Update 1 or later).

UWP using Entity Framework 7 – use SQLite for persistent storage: EF7Sample

UWP using SQLite-NET, second example using SQLite-WinRT NuGet.
The SQLite-NET sample also demonstrates accessing a SQLite database from a background task (Click the Background Task link on the command bar menu to navigate to a page where you can register and signal (i.e. start) the background task).
*Check the SQLite Extension SDK references in the foreground projects. The samples were built with 3.10.0 but if you have a different SQLite version installed, you will need to remove and then re-add the SQLite extension reference*
SQLiteSamplesUWP

Baby steps into the Internet of Things #1

First Steps #1: What is the Internet of Things?

You might have noticed a bit of chatter recently about something called the “Internet of Things” or IoT for short. It’s a classical industry hype buzz – everyone talking about it, a lot of people worried that they’re going to miss out on this ‘next big thing’ if they’re not careful. In fact, Gartner has called it as currently right at the peak of  the curve of ‘hype, and unrealistic expectations’ on their classical curve of technology adoption – which means the inevitable ‘trough of disillusionment’ is only just around the corner!

But this thing is real, and it’s exciting. I can make an honest claim to have been in IoT for many years, since I have been working for many years writing apps with the .NET Compact Framework on small processors running Windows CE in embedded applications, and on small handheld computers running Windows Mobile or Windows CE and more recently Windows Phone.

But all that is a niche (but very interesting) area using relatively pricey computers, and the real reason for the excitement around IoT is that we can finally start to deliver on having really cheap computers and sensors incorporated into ‘things’ around us, delivering data that we can use to create really interesting solutions. And I want to be in on this too, so I’ve started to explore it a bit and I’m going to document my ‘journey’ in a short series of blog posts – this is the first.

Enter ‘The Makers’

So I get that in any IoT solution, there are ‘things’ which are gathering data. These ‘things’ can be very simple, such as PIR movement detectors, light sensors, temperature sensors or GPS locators, or more sophisticated such as NFC readers or facial recognition solutions to detect when a specific person has entered a location.

SesnorsEverywhere

These smart things can be modelled and tested out using development boards where you can prototype this basic information gathering. There are a number of cheap electronics development boards out there, such as Arduino, Raspberry Pi, Netduino, Intel Galileo to mention just a few, which electronics-savvy people can wire up to simple stuff such as sensors, LEDs, visual displays, buzzers or sounders, or to more sophisticated devices such as motors and actuators to do cool stuff. And these boards can easily be networked to the internet or to other things where they can send the data that they gather. The people who work with these development boards are collectively known as ‘makers’ and there’s a lively community out there, all helping each other to make cool stuff, and readily giving help to beginners.

For me – this is perfect! Way back in the mists of time, I studied at Manchester University and my degree course was Electronics. But on my course, I realised I was more interested in this relatively new industry called Computer Programming than I was in soldering, so after I got my degree, I’ve been working in software ever since and hardly done anything with electronics. Which seems a bit of a shame – so now I can go back to my roots, so to speak, and combine both a bit of electronics with my love of software solutions and hopefully build something exciting!

If you have no interest in electronics, don’t worry. You don’t have to know how to use a soldering iron to get involved with the IoT. This thing will be huge, and the cheap electronics that are embedded in the ‘things’ is just one part of it. Sure prototyping electronics boards is one part of the whole process, but once the prototype sensor bit of it has been perfected using a development board, then some factory somewhere will manufacture the production version of it which will end up built into your smart refrigerator, your entry door, your environmental sensor – or whatever it is. Then the data starts flowing and it’s time to build a solution that makes use of that data.

So, what does an IoT solution look like?

This was the first question I asked, as a beginner in this field. I had an idea that there was small electronics boards in there. The classical example is the smart refrigerator, which warns you when your milk has been in the fridge too many days so you can throw it out, or which automatically adds items to your electronic shopping list as you use them up. OK – but how would that actually work?

How would a smart refrigerator actually work?

Well, that kind of scenario isn’t easy. For starters, the smart refrigerator would have to ‘know’ what items were in your fridge. I guess you could try to incorporate a barcode scanner in the door so that each time an item goes in and out, the user would have to scan the code on the packaging. Hmmm – no, that’s no good: too intrusive for the user. We could get an industry consensus-thing going so that every food manufacturer put an NFC tag on their packaging! Yeah, great idea, not going to happen.

So today, you’d have to sell your smart refrigerator with a big bag of smart tags, which the user would attach to products when they bought them, and which would be separated into different categories, maybe, milk, butter and any other specific products that the user wants to track. That’s probably OK since the user chooses which products to track, they stay in control and probably the only annoyance would be when the user accidentally throws out another smart tag with the trash and periodically has to purchase some more? Although – actually these pretty dumb tags could be passive NFC and maybe they would be cheap enough to throw away, and maybe throwing them away would be a good thing, since that might avoid mistakes like your fridge telling you your milk has gone off when actually all you’ve done is to re-use a tag but attached it to a new milk carton.

Alright, so now we’ve tracked the products and now the fridge can gather data about products going in and out of your fridge. What should it do with that data?

What to do with all that data?

This is where you start to see the anatomy of a complete IoT solution appear. The fridge itself could have software running in it to make decisions based on the data that it has – but then it would also have to be updated when software updates become available. And it would probably also have to offer some kind of UI so that the end user can configure their fridge. This doesn’t sound like a flexible, extensible solution.

So wouldn’t it be better for the fridge to simply take all that data and transmit it up to a database in the cloud, where the smart stuff would happen? Jobs would run in the cloud to analyse the stream of event data coming from smart refrigerators. The cloud would also host the end-user website where they can configure their fridge and determine what kind of alerts should be raised and what actions to take when their milk has been in the fridge for four days, and because it is in the cloud, it can easily be updated with new features.

The fridge itself acts as a gateway to ship the raw data up to the cloud, but it also acts as a consumer of the outputs of the processing and has buzzers and a display panel to alert the user to messages which have come down from the cloud as a result of the analysis of the raw data. But think how easy it would be to also send alerts to the users’ smartphone, or email, or SMS or…?

IoT-simplified

This sounds good! Now we’ve got a good solution for this. Good for the end-user – but also good for the manufacturer of the smart fridge who not only offers a good service to consumers, but can also get diagnostic messages from their smart refrigerators through the cloud, and also gets some potentially valuable usage data about what users are putting in their fridges which could guide their future product development and could possibly be sold onto other organisations– with end user permission of course.

The basic anatomy of an IoT solution

So with that I think I’ve figured out the basic layout of an IoT solution: sensors gather data which you ship to the cloud where you process it and publish the results to some consumer(s) – which could be a website, a smartphone or computer, or some other device.

Next thing for me is to take a closer look at the data gathering bit of all this and to get my hands dirty with a development board. I’m going to get myself a development board and have my first experiences of being a ‘maker’! That’ll all be in my next post!

Windows Phone 8.1 Smart card Reader sample code

I have had quite a few queries from developers who watched the Building Apps for Windows Phone 9.1 Jump Start videos and are interested in the low-level smart card reading that I showed in module 15: Sensors and Proximity: NFC and Bluetooth in which I demoed an app that read data off a bank debit card. You can find all the samples we showed in the Jump Start at http://aka.ms/wp81samples but the smartcard sample is missing from there.

There were two reasons for that: firstly, there have been no phones available that can run it! Until now that is. Low-level card reading needs a new NFC chipset that is only available in the newest phones that are only now coming to market, such as the Lumia 730 and Lumia 830. The device I used in the video was an engineering prototype. And the second reason for not publishing it was that at the time we published the code samples, I wasn’t sure we had the go-ahead from the card company whose card we used.

Anyway, now the phones are becoming available and the NFC Smart card team at Microsoft have published their ‘official’ sample up on MSDN code samples: NFC Smart Card Reader PC/SC Library . The Home page of the project explains it all perfectly, so I’ll just repeat a shortened version here:

“Example code and library for raw communication to NFC tags with Windows Phone 8.1, but requires a device with support (Lumia 830 & 73x)
NOTE: This code will work only on a Windows Phone running 8.1 or higher, and with an NFC driver that supports smart card reader mode such as the NXP PN547 which is available on the Lumia 730 and Lumia 830 (it will NOT work on a Lumia 930 or any existing hardware, even after updating to 8.1). See inside the MainPage.GetDevices() function for how to detect at runtime whether a device supports this functionality.

A new feature in Windows Phone 8.1 (on specific new hardware devices) is to be able to directly send raw commands to external plastic NFC cards, previously the phone could only read/write NDEF messages to NDEF-formatted tags but now you can talk to things like transit passes etc. For NXP PN547-based devices NFC cards such as MIFARE Classic/Ultralight/DESfire, ISO7816-4/APDU-based cards like JCOP, Jewel are supported as well as some limited support for ISO15693 and Felica. This project contains sample code only for MIFARE Classic/Ultralight/DESfire and Felica, but can be easily extended to support the others.”

So there you go! Go download this sample, get yourself one of the new phones, and start building interesting apps that interact with transit passes, security passes and take advantage of other smart card use scenarios!

Hey, Cortana – How can I integrate my app into you?

At the recent IFA 2014 show in Berlin, Microsoft announced some great new phones, the budget Lumia 730 and 735 and the ‘affordable premium’ phone the Lumia 830. Exciting new hardware – but the standout new feature for many was the announcement of ‘Hey, Cortana’, a new accessibility feature for the popular voice-controlled assistant which allows you to launch the app just by saying ‘Hey, Cortana’ at any time, avoiding the need to press the search button or the Live Tile as you do currently. This feature will be available on the high-end devices with the Snapdragon 800 processor, such as the Lumia 930, 1520 and Icon, and will be shipped as part of the Lumia ‘Denim’ firmware update coming to phones in Q4 2014. Cortana has been getting many plaudits – rightly so – and is currently available in the US, on beta in the UK and China, and will soon be rolling out as a beta soon to Australia, India and Canada, with more countries next year.

clip_image002

Speech control is part of the next wave of improvements of how we interact with our devices, part of a loose grouping of technologies often called NUI (Natural User Interface), meaning human to device interface methods that are effectively invisible, i.e. ‘natural’, as distinct from artificial ways of interacting with a device such as pressing a button which have to be learnt.

So having Cortana capable of passive listening so you can just talk to it whenever the fancy takes you and wake it up is another small step towards making speech interaction with our devices easier and more natural. It removes one more piece of friction – the need to tap on a button or tile to start the voice interaction – that could prevent users choosing voice as their favoured means of interaction with their devices rather than one only used by enthusiasts.

Integrating Apps into Cortana

So that’s great, now we can say ‘Hey, Cortana’ to invoke the built-in digital assistant features of Cortana and get it to tell us upcoming events in our calendar, set new reminders, do searches etc. But what if you want to allow users to interact with your app in a similar way, use Cortana as a conduit for speech interaction with your app? Say something like: “Hey Cortana” to invoke Cortana, and then say “MSDN find ‘Windows Phone Voice Commands’” to invoke the MSDN Voice Search app.

Well, that’s not too hard! In fact, we’ve had the technology even in Windows Phone 8.0, before Cortana was a twinkle in Halo’s eye. Back in the Windows Phone 8.0 Jump Start videos, we recorded one session all about voice commands (http://channel9.msdn.com/Series/Building-Apps-for-Windows-Phone-8-Jump-Start/Building-Apps-for-Windows-Phone-8-Jump-Start-13-Speech-Input-in-Windows-Phone-8 ) which explained how to create a Voice Command definition file to allow you to launch apps with speech. It’s that same technology that is used to launch apps through speech from Cortana.

Although it was quite easy to implement, not that many developers added voice command activation to their 8.0 apps and very few users turned to voice as their favoured method of launching their favourite app. Why not? Discoverability! To activate voice recognition on a Windows Phone 8.0 device, you had to press and hold the Windows button. Most users only ever discovered this by accident, if at all! So as a consequence, it never gained wide usage.

That’s why Cortana is so cool – everyone knows that it is the voice portal on a Windows Phone app. It’s no longer down to knowing about some unnatural user interaction mechanism – a tap and hold on the Windows key – but now you invoke Cortana by a single tap on the Search button, or by tapping on the Live Tile, or even by launching it from the Apps list. Or – better by far – by simply saying ‘Hey, Cortana’ on a top-end phone with the new Lumia Denim software. [Note that in countries that do not have Cortana yet, you can still launch apps by voice command

Voice Command Recognition in Windows Phone 8.0

The Voice Commands implementation in Windows Phone 8.0 had another limitation apart from discoverability: it was a little too inflexible. With Windows Phone 8.0 Voice Commands, the full text of the commands themselves needed to be known ahead of time—this made them difficult to use for searching, social networking, or anything else with a large vocabulary.

For example, a Windows Phone 8.0 voice command definition looks like this:

    <CommandSet xml:lang="en-gb" Name="UKenglishCommands">
        <!-- The CommandPrefix provides an alternative to your full app name for invocation -->
        <CommandPrefix> MSDN </CommandPrefix>
        <!-- The CommandSet Example appears in the global help alongside your app name -->
        <Example> search </Example>

        <Command Name="MSDNSearch">
            <!-- The Command example appears in the drill-down help page for your app -->
            <Example> do a search </Example>

            <!-- ListenFor elements provide ways to say the command, including references to 
            {PhraseLists} and {PhraseTopics} as well as [optional] words -->
            <ListenFor> [do a] Search [for] {*} </ListenFor>
            <ListenFor> Find {*} </ListenFor>
            <ListenFor> Look [for] {*} </ListenFor>

            <!--Feedback provides the displayed and spoken text when your command is triggered -->
            <Feedback> Searching MSDN... </Feedback>

            <!-- Navigate specifies the desired page or invocation destination for the Command-->
            <Navigate Target="MainPage.xaml" />
        </Command>
    </CommandSet>

Every command must start with the CommandPrefix – ‘MSDN’ in this case. Thereafter, the ListenFor elements define the different variants for a specific speech command, and these may include optional words if they are enclosed in square brackets. There is the ability to allow for free speech, which is represented by {*} but that still had to be preceded by some fixed speech components in the command. So for this example, all of the following will trigger the voice command:

  • Search xyz
  • Search for xyz
  • Find xyz
  • Look xyz
  • Look for xyz

But the following will *not* be recognised because it doesn’t fit into one of the ListenFor definitions:

  • I want to know about xyz

Recognizing natural language in Windows Phone 8.1

The Voice Command Definition for 8.1 allows more flexibility:

<?xml version="1.0" encoding="utf-8"?>

<!-- Be sure to use the new v1.1 namespace to utilize the new PhraseTopic feature -->
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.1">
    <!-- The CommandSet Name is used to programmatically access the CommandSet -->
    <CommandSet xml:lang="en-gb" Name="UKenglishCommands">
        <!-- The CommandPrefix provides an alternative to your full app name for invocation -->
        <CommandPrefix>MSDN</CommandPrefix>
        <!-- The CommandSet Example appears in the global help alongside your app name -->
        <Example> find 'Windows Phone Voice Commands' </Example>

        <Command Name="MSDNSearch">
            <!-- The Command example appears in the drill-down help page for your app -->
            <Example> find 'how to install CommandSets' </Example>

            <!-- ListenFor elements provide ways to say the command, including references to 
            {PhraseLists} and {PhraseTopics} as well as [optional] words -->
            <ListenFor> Search </ListenFor>
            <ListenFor> Search [for] {dictatedSearchTerms} </ListenFor>
            <ListenFor> Find {dictatedSearchTerms} </ListenFor>
            <ListenFor> Find </ListenFor>

          <!--Feedback provides the displayed and spoken text when your command is triggered -->
            <Feedback> Searching MSDN... </Feedback>

            <!-- Navigate specifies the desired page or invocation destination for the Command-->
            <Navigate Target="MainPage.xaml" />
        </Command>

        <Command Name="MSDNNaturalLanguage">
            <Example> I want to go to the Windows Phone Dev center </Example>
            <ListenFor> {naturalLanguage} </ListenFor>
            <Feedback> Starting MSDN... </Feedback>
            <Navigate Target="MainPage.xaml" />
        </Command>

        <PhraseTopic Label="dictatedSearchTerms" Scenario="Search">
            <Subject> MSDN </Subject>
        </PhraseTopic>

        <PhraseTopic Label="naturalLanguage" Scenario="Natural Language">
            <Subject> MSDN </Subject>
        </PhraseTopic>

    </CommandSet>
</VoiceCommands>

Notice the following ListenFor elements:

        <Command Name="MSDNSearch">
           …
            <ListenFor> Search [for] {dictatedSearchTerms} </ListenFor>
           …
        </Command>
        <Command Name="MSDNNaturalLanguage">
            …
            <ListenFor> {naturalLanguage} </ListenFor>
            …
        </Command>

Both of these elements contain a label inside curly brackets which refers to the PhraseTopic elements defined towards the bottom. First dictatedSearchTerms is defined as:

        <PhraseTopic Label="dictatedSearchTerms" Scenario="Search">
            <Subject> MSDN </Subject>
        </PhraseTopic>

A PhraseTopic is a new element in 8.1 which specifies a topic for large vocabulary definition. What this means is that the voice input goes up to the cloud where it is recognized against the same large vocabulary of words that Cortana uses for its built-in functions. The Scenario attribute is used to guide the recognition. Valid values are “Natural Language”, “Search”, “Short Message”, “Dictation” (the default), “Commands”, and “Form Filling”. The Subject child element specifies a subject specific to the parent PhraseTopic’s Scenario attribute to further refine the relevance of speech recognition results.

So the command that uses this ListenFor definition allows free speech searches in MSDN.

The second ListenFor element has no fixed command terms at all:

<ListenFor> {naturalLanguage} </ListenFor>

Where naturalLanguage is defined as:

        <PhraseTopic Label="naturalLanguage" Scenario="Natural Language">
            <Subject> MSDN </Subject>
        </PhraseTopic>

Effectively, this defines an ‘anything goes’ command, so this can recognize a query such as “I want to know about xyz”. Consider this a free-form alternative to the other, more rigid, command labelled MSDNSearch where the ListenFor elements define fixed components of the speech command (‘Find’, ‘Search’) that must be present for recognition to succeed.

But how then does the app know what it should do when “Natural Language” speech input is recognized? There’s no magic here, you have to code your app to search for elements in the recognized speech that have relevance to that apps problem domain. The MSDN Voice Search sample app (link in the ‘Further Reading’ section at the end) is coded to recognise variants on ‘Take me to the Windows Dev Center’ and if that isn’t found falls back to a default of a search on Bing for whatever speech input was recognized. We’ll look at how this is done in just a moment.

Handing Speech Commands in your app

I’m not going to go into all the coding techniques used to handle voice activation here. Watch the BUILD video (link below in ‘Further Reading’) for an excellent overview of handling voice activation in both Silverlight and Windows Runtime apps, and download the sample app (link also below). But I will just call out how to handle Natural Language input.

The sample app contains the following code for handling this:

        /// <summary>
        /// Given a query string from the user, attempts rudimentary "natural language" string processing in an
        /// effort to derive a specific, curated action from the text; if a match is found, that curated action
        /// is taken. If not, the string is unhandled and should be handled elsewhere.
        /// </summary>
        /// <param name="query"> the query to attempt processing and action upon </param>
        /// <param name="actSilently"> whether or not to only take actions without audio feedback </param>
        /// <returns></returns>
        private bool TryHandleNlQuery(string query, bool actSilently)
        {
            // There are a variety of ways to say things like "I want to go to Windows Phone Dev Center"; let's load
            // some alternatives for the key components of this.
            string[] intentMarkers = AppResources.NaturalLanguageCommandIntentMarkers.Split(new char[] { ';' });
            string[] wpDevCenterNames = AppResources.WPDevCenterNames.Split(new char[] { ';' });

            int intentIndex = -1;
            int destinationIndex = -1;

            Uri destinationUri = null;
            string confirmationTts = null;

            // First we'll try to find a match for the "intent marker," e.g. "go to"
            foreach (string marker in intentMarkers)
            {
                intentIndex = query.IndexOf(marker, StringComparison.InvariantCultureIgnoreCase);
                if (intentIndex >= 0)
                {
                    break;
                }
            }

            if (intentIndex >= 0)
            {
                // Now we'll try to figure out a destination--if it comes after the intent marker in the string, we'll
                // store the destination and spoken feedback.
                foreach (string wpDevCenterName in wpDevCenterNames)
                {
                    destinationIndex = query.IndexOf(wpDevCenterName, StringComparison.InvariantCultureIgnoreCase);
                    if (destinationIndex > intentIndex)
                    {
                        destinationUri = new Uri(AppResources.WPDevCenterURL);
                        confirmationTts = AppResources.SpokenWPDevCenterSsml;
                        break;
                    }
                }
            }

            // If we found a destination to go to, we'll go there and--if allowed--provide the corresponding spoken
            // feedback.
            if (destinationUri != null)
            {
                if (!actSilently && (confirmationTts != null))
                {
                    StartSpeakingSsml(confirmationTts);
                }

                StartBrowserNavigation(destinationUri);
            }

            // If we found a destination, we handled the query. Otherwise, it hasn't been handled yet.
            return (destinationUri == null); ;
        }

That’s a lot of code, and essentially all it does is look for a number of different variants on ‘I want to go to the Windows Dev Center’. Key to understanding it is knowing that in the Resources file, the command intents and variations for the dev center name are defined:

NaturalLanguageCommandIntentMarkers: show me;go to;take me to;bring up;navigate to;launch
WPDevCenterNames: Phone Dev Center;Phone Developer Center

So all the code shown above actually does is to search for a match on the recognized text for one of the NaturalLanguageCommandIntentMarkers, followed by one of the options for WPDevCenterNames. If it matches then the app takes you to the Windows dev center. If you follow through on the sample code, you’ll see that if no match is found, then the default fallback action is just to invoke a Bing search for the recognized text.

Like I said – there’s no magic here. You just have to write code to analyse the recognized speech for strings that make sense for that app, and this could be quite a lot of code!

Parting thoughts…

Before I close, three further thoughts on Speech enabling your apps:

First, make sure you include Command definitions in your vcd file for all languages you want to support. The sample app has commands defined only for US English:

    <!-- The CommandSet Name is used to programmatically access the CommandSet -->
    <CommandSet xml:lang="en-us" Name="englishCommands">

which means that when you run it on a UK English or Australian English device, the voice commands don’t work! You have to define for each individual language you want, which could get a bit tedious. But be warned – it caught me out!

Secondly, make sure you detect when the user has typed into Cortana rather than used speech. If they typed in, don’t go speaking back to them – they typed quite possibly because they were in a meeting, so it would not be helpful to start speaking out sports results or whatever! The sample app and the BUILD video show you how to do this.

And thirdly, your user has to run the app at least once to run the code to register the speech commands with the system. Unfortunately, speech commands do not get activated just by installing the app. So the first time the user runs the app, that might be a good time to pop up a ‘Did you know?’ screen to tell them about the speech commands your app supports. All helps towards the users’ discoverability of these kind of features.

Further reading

· Video: Integrating Your App into the Windows Phone Speech Experience http://channel9.msdn.com/Events/Build/2014/2-530

· Sample: MSDN Voice Search for Windows Phone 8.1 http://code.msdn.microsoft.com/MSDN-Voice-Search-for-95c16d92

· Speech team blog post: Integrating Store Apps with Cortana in Windows Phone 8.1 http://blogs.bing.com/search/2014/04/14/integrating-store-apps-with-cortana-in-windows-phone-8-1/

Voice command element and attribute reference (Windows Phone Store apps using C#/VB/C++ and XAML)
http://msdn.microsoft.com/en-us/library/dn630431.aspx

Windows Apps JumpStart LIVE Events Series

We’re starting a new series of events for UK developers that will help you on the journey from initial interest in developing apps for Windows 8 and Windows Phone right the way through to publishing an app to the Windows Store. You can join into the series at whichever point makes sense to you, so whether you’ve never opened Visual Studio before or written a line of C#, or you’re a developer who has already written software using out tools and languages but are new to apps, we can help you get to that point where you’re publishing your first app into our store. And we’re going to repeat the series of events every quarter, so you can jump on and off the series and attend events at whatever pace makes sense to you.

We start with two one-hour webinars run back-to-back:

Webinar 1: Windows Apps JumpStart LIVE Series – Kick Starter Webinar for the non C#/XAML developer: This one hour webinar is aimed at developers who are completely new to Visual Studio, XAML, C# and .NET. This will give you what they need to get started with downloading Visual Studio, a little bit of knowledge of C# and the structure of a XAML-based app.

Webinar 2: Windows Apps JumpStart LIVE Series – JumpStart Webinar: Building Apps with C# and XAML: This one hour webinar follows on straight away after the previous one. It is aimed at developers who are new to Windows Store app development, but who have some experience of developing software using .NET, or for those who have attended the previous webinar.

Following on from the webinars, we have a one-day dev camp:

Dev camp: Windows Apps JumpStart LIVE Series: Windows Apps Dev Workshop
This one day event will be help at Microsoft’s offices near Victoria Station, London, running 9:30 – 18:00. At this workshop, you will learn how to program Windows and Windows Phone apps, including universal apps that run on both. There will be Hands-on Labs available, and you can get tips and advice from Microsoft experts and from peers in the developer community.

And following on from that, we have a one-day Publish hackathon:

Publish event: Windows Apps JumpStart LIVE Series: Publish Hackathon
This informal one-day event will also be held at Microsoft’s building in London, running 9:00 – 21:00 on a Saturday. This is a fun 12 hour event encouraging developers to bring their existing projects (apps/cloud projects) to polish and publish through to the last mile with help from Microsoft and community experts.

First events are coming up fast!:

Come along and join us – and if not this quarter, remember we are repeating these events throughout the year so you can join us when you can!

Push Notifications made easy: Using Windows Azure Notification Hubs with Windows Phone

When I built the Windows Phone 8 Jump Start training, one of the sessions was on Push Notifications. The demo app I used was an update of an old favourite, a Weather app which first saw the light of day a few years ago with the initial demos built for Windows Phone 7.0. In the demo, you have a phone app and there is also a WPF desktop app.

The WPF desktop app allows the ‘operator’ to select a city, temperature and weather conditions and then generate tile, toast and raw notification messages to communicate the weather forecast to the phone, but the WPF app is actually more complex than that and actually fulfils two key roles in a push notifications scenario, as it also hosts a WCF service and maintains a list of subscribers, simulating the cloud service that you would build for a real Push Notifications solution.

PushBasics

In summary, the phone registers for push notifications1 and receives a unique URI (the channel URI) which it then sends off to your cloud service2 (the WCF service) and the cloud service maintains a list of all subscribers for this app. Then the backend logic (the WPF UI in this demo) reads the list of subscriber channel URIs when it has something to communicate to the phone and sends the tile/toast/raw payloads to the subscriber URIs3. The subscriber URIs are actually endpoints hosted on MPNS (Microsoft Push Notification Services), so MPNS actually takes care of delivering the payloads to the phone4.

Fine – except that sample is difficult to setup and run, since you have to run Visual Studio as administrator otherwise the WCF self-hosting doesn’t work, you have to open port 8000 in your firewall because that is the port the WCF service listens on, and you have to edit the phone app code to set the correct hostname of the PC where the WCF service is running. Those are all implementation difficulties, but the real crime of this demo is that it’s too simplistic a demo and gives the impression that writing a Push Notifications solution is easier than it really is. In a real world app, when you post to MPNS you get back a response that gives you information about the state of the phone app subscription. There’s a whole page in the documentation about what you might get back: Push Notification Service response codes for Windows Phone and even if you ignore most of that, at the very least you should be removing subscribers from your list if you get a 404 Not Found response back, indicating that the subscription channel URI you posted to is no longer valid.

In fact, writing backend logic to send push notifications and correctly react to the Push Notification Service response codes is hard. And if you want to send push notifications to more than one client platform, such as Windows 8, iOS and Android, it’s incredibly hard.

That is – unless you use Windows Azure Notification Hubs Smile

Windows Azure Notification Hubs

Windows Azure Notification Hubs is a service running in Windows Azure that makes it really easy to send push notifications to mobile clients. You can read an overview here, but in essence what it does is remove the need to manage the subscriber list and create your own subscription WCF service from your backend logic and provide an easy to program API for sending notifications – and not only to Windows Phone, but to Windows 8, iOS and Android clients too.

But the benefits of Notification Hubs are not only for those building a cross-platform solution. Even if you are building a push notifications app for a single type of client, using this technology will greatly ease the creation of your solution.

I thought it would be interesting to do a new version of the Weather sample, but using Windows Azure Notification Hubs to make a comparison between the two ways of doing it, so for the rest of this post, I will walk you through what I did (download the sample code from the link at the bottom of this post).

From an architectural point of view, the main thing is that the custom WCF service and subscriber list disappears completely. Great, no need to build and host a custom web service in order to build a push solution! Instead, the notification hub fulfils that purpose and acts as a gateway between your backend logic and MPNS.

PushNotificationsWithWANH

Configuring the Notification Hub

First stage in building this is to configure the notification hub in Windows Azure, which is all done using the portal. You need to have a Windows Azure account and then go to the portal at http://manage.windowsazure.com.

  1. Click +NEW at the bottom of the screen.
  2. Click App Services, then Service Bus, then Notification Hub, then Quick Create.
  3. Type a name for your notification hub – I chose weathernotificationhub, select your desired region – West Europe for me, then click Create a new Notification Hub.
    WANHcreate
  4. You will now see the Service Bus namespace created screen.
    NotificationHubCreated

    Now, before you leave this screen, click on the Install the Windows Azure SDK (includes Service Bus client libraries) link to make sure you’ve got the client libraries installed in Visual Studio that you will need in both the phone app and also the desktop app.

  5. Click Connection Information at the bottom. Take note of the two connection strings – you will need them in a moment.
    AccessConnectionInformation

That’s it. Your Windows Azure Notification Hub is configured and ready to go.

Connecting the Phone App to Windows Azure Notification Hubs

Now let’s turn to the phone app. There aren’t many changes we need to make from the phone app we used in pre-notification hubs days. It still needs to register for push notifications using Microsoft.Phone.Notifications.HttpNotificationChannel and register for toast, tile and/or raw notifications. And when the app opens the channel, it will get a new Channel Uri reported to it when the ChannelUriUpdated event fires.

In the original app, it was at this point that you sent the channel Uri off to the WCF service hosted in the Weather Service desktop app, but instead we now send that off to our Windows Azure Notification Hub. To do this, you need to add a reference to the client libraries using the WindowsAzure.Messaging.Managed NuGet package.

In the Visual Studio menu, click Tools, then Library Package Manager, then Package Manager Console. Then, in the console window type:

Install-Package WindowsAzure.Messaging.Managed

and press Enter.

Now in the phone app, in MainPage.xaml.cs, we add the using Microsoft.WindowsAzure.Messaging using statement at the top, and then modify the ChennelUriUpdated event handler as follows:

async void httpChannel_ChannelUriUpdated(object sender, NotificationChannelUriEventArgs e)
{
    Trace("Channel opened. Got Uri:n" + httpChannel.ChannelUri.ToString());
    Dispatcher.BeginInvoke(() => SaveChannelInfo());

    Trace("Subscribing to channel events");
    await SubscribeToServiceAsync();
    SubscribeToNotifications();

    Dispatcher.BeginInvoke(() => UpdateStatus("Channel created successfully"));
}

The SubscribeToServiceAsync() method is a new version of SubscribeToService() in the original demo. Instead of calling the WCF service (the commented out code), it now instantiates a Microsoft.WindowsAzure.Messaging.NotificationHub instance, which takes two arguments: the name of my notification hub (weathernotificationhub) and the connection string called DefaultListenSharedAccessSignature that you get from the Connection Strings display in the portal (step 5 in the previous section).

private async Task SubscribeToServiceAsync()
{
    try
    {
        var hub = new NotificationHub(
            "weathernotificationhub",
            "Endpoint=sb://weathernotificationhub-ns.servicebus.windows.net...[your value] ...");

        Registration registration = 
            await hub.RegisterNativeAsync(httpChannel.ChannelUri.ToString());
        Trace("Registered with Azure Notification Hub, Registration Id:" 
            + registration.RegistrationId);

        Dispatcher.BeginInvoke(() => 
            UpdateStatus("Registered with Windows Azure Notification Hub"));
    }
    catch (RegistrationAuthorizationException rEx)
    {
        Trace(rEx.Message);
        throw;
    }
}

That’s it as far as the phone client goes. Everything else stays the same: push notifications are delivered by the notifications infrastructure and handled in exactly the same way. Windows Azure Notification Hubs simply replaces your own subscription service, as far as the phone client goes.

Sending Notifications From Your Backend Logic

This is where the true benefit of Windows Azure Notification Hubs shines through. You no longer have to create your own web service with table storage to receive the channel URIs from your phone client subscribers, nor do you have to write logic to examine the status returns you get back from MPNS when you post notification messages and prune out stale registrations from the subscriber lists. You don’t need to write logic to post to each subscriber channel URI individually, just one call to the notification hub will cause the message to be sent out to all your clients. And if your backend logic needs to support Windows 8, iOS or Android clients as well as Windows Phone, the benefits of  Windows Azure Notification Hubs multiplies a hundred-fold.

Getting Registration Details from the Hub

In the Weather app, I started by deleting everything to do with the WCF registration service that we used in the previous version. Then fixed up the UI where it used to display things like “Waiting for connection…”. One thing the original did, was show the current count of registrations to the registration service. To do the same with Notification Hubs, you can call the NotificationHubClient.GetAllRegistrationsAsync(int32 top) method which returns details of all registrations limited to the top ‘n’ by the value you pass in the argument. In order to use the NotificationHubClient object, you need to add a reference to the client libraries using the WindowsAzure.Messaging.Managed NuGet package just the same as we did with the phone app.

After adding the NuGet package to my sample app, we show the number of clients on the screen using the following method:

private async Task UpdateRegistrationsCountAsync() { NotificationHubClient hub = NotificationHubClient.CreateClientFromConnectionString( "Endpoint=sb://weathernotificationhub-ns.servicebus.windows.net/;…", "weathernotificationhub"); var registrationsCount = await hub.GetAllRegistrationsAsync(Int32.MaxValue);

await Dispatcher.BeginInvoke((Action)(() =>
{ txtRegistrationsCount.Text = registrationsCount.Count().ToString(); })
);
}

The NotificationHubClient constructor takes two arguments: the first is the connection string you get from the Notification Hub Connection Strings page in the Windows Azure portal, but unlike the phone client which used the DefaultListenSharedAccessSignature, this time you need the DefaultFullSharedAccessSignature as this app is doing full interaction with the notification hub, such as posting notifications. The second arg is the name of your notification hub.

Sending Toast and Tile Push Notifications

Now to the main purpose of the app. Most of the logic around formatting and sending notification messages is encapsulated into the NotificationSenderUtility class in my sample. For example, this contains the following method to format the XML to send for a Windows Phone Toast notification:

private static string prepareToastPayload(string text1, string text2)
{
    // Create encoding manually in order to prevent
    // creation of leading BOM (Byte Order Mark) xFEFF at start
    // of string created from the XML
    Encoding Utf8 = new UTF8Encoding(false); // Prevents creation of BOM
    MemoryStream stream = new MemoryStream();
    XmlWriterSettings settings = new XmlWriterSettings() 
                { 
                    Indent = false,
                //   Encoding = Encoding.UTF8    !!NO-> adds Unicode BOM to start
                    Encoding = Utf8,    // Use manually created UTF8 encoding
                };
    XmlWriter writer = XmlWriter.Create(stream, settings);
    writer.WriteStartDocument();
    writer.WriteStartElement("wp", "Notification", "WPNotification");
    writer.WriteStartElement("wp", "Toast", "WPNotification");
    writer.WriteStartElement("wp", "Text1", "WPNotification");
    writer.WriteValue(text1);
    writer.WriteEndElement();
    writer.WriteStartElement("wp", "Text2", "WPNotification");
    writer.WriteValue(text2);
    writer.WriteEndElement();
    writer.WriteEndElement();
    writer.WriteEndDocument();
    writer.Close();

    return  Encoding.UTF8.GetString(stream.ToArray());
}

WARNING: While building this, I discovered that Windows Azure Notification Hubs are quite a bit fussier about the XML you send it than MPNS actually is. The original version of the code used the standard Encoding.UTF8 in the XmlWriterSettings (commented out in the code above) which results in an invisible Byte Order Mark (BOM) character being inserted at the front of the string, which causes Windows Azure Notification Hubs to barf when you send it that. The code above creates the XML without the BOM at the front which works fine. To read more about this, see my blog post BOMbin’ the L (aka Wierd Errors with XmlWriter).

Having prepared the payload, to send a Toast or Tile is simply a case of calling the NotificationHubClient.SendMpnsNativeNotificationAsync(string payload) method. The implementation in the attached sample is slightly more complex than this, but essentially it’s the following:

    NotificationHubClient hub = NotificationHubClient.CreateClientFromConnectionString(
        "Endpoint=sb://weathernotificationhub-ns.servicebus.windows.net/;...",
        "weathernotificationhub");

    NotificationOutcome outcome = await hub.SendMpnsNativeNotificationAsync(payload);

Sending Raw Notifications

The other thing the original sample does is send raw notifications directly to the app (if it happens to be running of course). The body of a raw payload is entirely up to the app developer, since unlike toast and tile notifications that may be processed by the Push Notifications handler in the phone OS, raw notifications are only ever handled by the app. So in the new version of this sample, the code building the raw notification payload is almost the same as before:

private static string prepareRAWPayload(string location, string temperature, string weatherType)
{
    // Create encoding manually in order to prevent
    // creation of leading BOM (Byte Order Mark) xFEFF at start
    // of string created from the XML
    Encoding Utf8 = new UTF8Encoding(false); // Prevents creation of BOM
    MemoryStream stream = new MemoryStream();
    XmlWriterSettings settings = new XmlWriterSettings()
    {
        Indent = false,
        //   Encoding = Encoding.UTF8    !!NO-> adds Unicode BOM to start
        Encoding = Utf8,    // Use manually created UTF8 encoding
    };
    XmlWriter writer = XmlTextWriter.Create(stream, settings);

    writer.WriteStartDocument();
    writer.WriteStartElement("WeatherUpdate");

    writer.WriteStartElement("Location");
    writer.WriteValue(location);
    writer.WriteEndElement();

    writer.WriteStartElement("Temperature");
    writer.WriteValue(temperature);
    writer.WriteEndElement();

    writer.WriteStartElement("WeatherType");
    writer.WriteValue(weatherType);
    writer.WriteEndElement();

    writer.WriteStartElement("LastUpdated");
    writer.WriteValue(DateTime.Now.ToString());
    writer.WriteEndElement();

    writer.WriteEndElement();
    writer.WriteEndDocument();
    writer.Close();

    return Encoding.UTF8.GetString(stream.ToArray());
}

I then had to do a bit of digging to figure out how to send this raw notification via Windows Azure Notification Hubs. One of the many methods of NotificationHubClient is SendNotificationAsync(Notification notification) which is a more generic Send* method than SendMpnsNativeNotificationAsync that we used before. It has the advantage that we can add HTTP headers to the request we send to MPNS, so we can define the X-NOTIFICATION header that a raw notification requires:

        // For raw notifications, have to use the more generic SendNotificationAsync() method
        Notification notification = new MpnsNotification(payload);
        notification.Headers.Add("X-NotificationClass", "3"); // Required header for RAW
        outcome = await hub.SendNotificationAsync(notification);

push end

Summary

Hopefully this has given you an introduction to building a push notification solution using Windows Azure Notification Hubs. There is plenty more to discover as I have only scratched the surface. For example, this sample sends notifications to all subscribers, but you can send to only a subset of subscribers using Tags. And of course, you can also send to Windows 8, iOS and Android clients, formatting the payloads for each client family yourself and then calling the SendWNSNativeNotificationAsync, SendAPSNativeNotificationAsync or SendGCMNativeNotificationAsync methods. An even easier way to do this, is to use the Templates feature, which is a neat way of sending a single generic message from your backend logic and allowing Windows Azure Notification Hubs to format the native message payload for each platform for you.

Here are some useful resources to allow you to dig deeper into Windows Azure Notification Hubs:

You should also be aware that Windows Azure Mobile Services uses Notification Hubs to send push notifications.

You can download the sample code described here yourself using the link at the bottom of this post. You will have to create your own notification hub and plug in your own hub name and connection strings, both in MainPage.xaml.cs in the phone project, and in MainWindows.xaml.cs in the desktop app – you should be able to find where easily enough.

 

BOMbin’ the L (aka Wierd Errors with XmlWriter)

[Title from a track on Come Find Yourself– Fun Lovin’ Criminals]

Have you experienced strange errors when working with XML generated by an XMLWriter? If so, this post may be helpful to you. I just spent a few hours trying to figure out why I was getting a weird error when I was sending some XML to the Windows Azure Notification Hubs client api (post on that coming up, just as soon as I have my sample working).

There’s a page here Get Started with Notification Hubs with tutorials on using this technology with Windows Store C#, Windows Phone, Android and iOS clients. In the Windows Phone tutorial, it walks through creating a Windows Phone client (needless to say) and a console app which simulates your backend logic that you use to send Tile, Toast or Raw notification messages to the mobile clients. In the instructions for the console app it includes the following:

In your Program class add the following method:

private static async void SendNotificationAsync()
{
    NotificationHubClient hub = NotificationHubClient.CreateClientFromConnectionString("<connection string with full access>", "<hub name>");
    string toast = "<?xml version="1.0" encoding="utf-8"?>" +
        "<wp:Notification xmlns:wp="WPNotification">" +
           "<wp:Toast>" +
                "<wp:Text1>Hello from a .NET App!</wp:Text1>" +
           "</wp:Toast> " +
        "</wp:Notification>";
    await hub.SendMpnsNativeNotificationAsync(toast);
}

Make sure to insert the name of your hub and the connection string called DefaultFullSharedAccessSignature that you obtained in the section “Configure your Notification Hub.” Note that this is the connection string with Full access, not Listen access.

Then add the following line in your Main method:

SendNotificationAsync();
 Console.ReadLine();

Easy enough! So, using this as my guide, I created my own app that followed these principles, the main difference being that instead of creating a string with the XML payload in it, I thought I’d use an XmlWriter instead, because that’s neater:

        private static string prepareToastPayload(string text1, string text2)
        {
            MemoryStream stream = new MemoryStream();
            XmlWriterSettings settings = new XmlWriterSettings() 
                                            { 
                                                Indent = false,
                                                Encoding = Encoding.UTF8   
                                            };
            XmlWriter writer = XmlWriter.Create(stream, settings);
            writer.WriteStartDocument();
            writer.WriteStartElement("wp", "Notification", "WPNotification");
            writer.WriteStartElement("wp", "Toast", "WPNotification");
            writer.WriteStartElement("wp", "Text1", "WPNotification");
            writer.WriteValue(text1);
            writer.WriteEndElement();
            writer.WriteStartElement("wp", "Text2", "WPNotification");
            writer.WriteValue(text2);
            writer.WriteEndElement();
            writer.WriteEndElement();
            writer.WriteEndDocument();
            writer.Close();

            return Encoding.UTF8.GetString(stream.ToArray());
        }

Wierd exception

When I ran this, I got the following exception:

System.ArgumentException: The payload is not in accepted XML format. The first node should be Tile/Toast. If want to send raw notification, please set “X-NotificationClass” to “3” in header.rn   at Microsoft.ServiceBus.Notifications.RegistrationSDKHelper.DetectMpnsTemplateRegistationType(String body, String errorMsg)rn   at Microsoft.ServiceBus.Notifications.MpnsNotification.OnValidateAndPopulateHeaders()rn   at Microsoft.ServiceBus.Notifications.Notification.ValidateAndPopulateHeaders()rn   at Microsoft.ServiceBus.Notifications.NotificationHubManager.BeginSendNotification(Notification notification, Boolean testSend, AsyncCallback callback, Object state)rn   at Microsoft.ServiceBus.Notifications.NotificationHubManager.<>c__DisplayClass1.<SendNotificationAsync>b__0(AsyncCallback c, Object s)rn   at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl(Func`3 beginMethod, Func`2 endFunction, Action`1 endAction, Object state, TaskCreationOptions creationOptions)rn   at System.Threading.Tasks.TaskF
actory`1.FromAsync(Func`3 beginMethod, Func`2 endMethod, Object state)rn   at Microsoft.ServiceBus.Common.Parallel.TaskHelpers.CreateTask[T](Func`3 begin, Func`2 end)… in c:\Development\AzurePushNotificationsDemo\NotificationSenderUtility\NotificationSenderUtility.cs:line 139

Now as it turns out, the error message here is rather misleading. “The first node should be Tile/Toast” seems to be referring to the expected XML format if you are sending toast messages using the (very useful) template capability of Notification Hubs, which allows you to send a generic message to Notification Hubs just once, and it will build the appropriate payloads for your actual clients, whether they are Windows Phone, Windows 8, Android or iOS, all of which have different payloads. But “The payload is not in accepted XML format” is a bit more interesting and forced me to take a look at the XML I was generating.

And here it gets really confusing. I swapped out the XmlWriter code and replaced it with just simply setting a string variable directly, similar to the original tutorial – that works! So what is different about the string generated by my prepareToastpayload method? I then put the two methods one after the other and stored the results in two different string variables which I then examined when stopped at a breakpoint – identical!! So, what gives?

Examining the string bytes

Of course, the strings WEREN’T identical, despite what the Visual Studio debugger was showing me! A bit of a web search led me to this useful post by Jon Skeet, author of the book “C# In Depth”. His post Strings in C# and .NET includes a warning about the Visual Studio debugger not showing you the whole story, and he includes some useful code for dumping the bytes of a string to output. I plugged in the suggested methods and the output revealed the following at the beginning of the XML string I was sending to SendMpnsNativeNotifiactionAsync():

image

Aha! What are these (Possibly non-printable) xFEFF bytes at the start? Next web search was for FEFF and I came upon a few posts explaining that hex FEFF is something called a BOM or Byte Order Mark, which is something that is usually inserted into UTF-8 encoded files that are stored on disk, which XML parsers can look for to figure out how to interpret the file contents. However, if you are formatting XML to send as payload in a web service call as we are here, then you don’t want it. Best post that explain this is a six year old blog by Rick Strahl, XmlWriter, Strings and Byte Order Marks .

Un-BOMbin’ the L

Rick gives the solution as well. It’s actually very difficult to stop the XmlWriter for outputting XML without a BOM, but you can get it to behave by creating a custom UTF-8 encoding where you can specify you don’t want a BOM by passing false into an override of the constructor, and then use that in your XmlWriterSettings object:

        private static string prepareToastPayload(string text1, string text2)
        {
            // Create encoding manually in order to prevent
            // creation of leading BOM (Byte Order Mark) xFEFF at start
            // of string created from the XML
            Encoding Utf8 = new UTF8Encoding(false); // Prevents creation of BOM
            MemoryStream stream = new MemoryStream();
            XmlWriterSettings settings = new XmlWriterSettings() 
                                            { 
                                                Indent = false,
                                           //   Encoding = Encoding.UTF8    !!NO-> adds Unicode BOM to start
                                                Encoding = Utf8,    // Use manually created UTF8 encoding
                                            };
            XmlWriter writer = XmlWriter.Create(stream, settings);
            writer.WriteStartDocument();
            writer.WriteStartElement("wp", "Notification", "WPNotification");
            writer.WriteStartElement("wp", "Toast", "WPNotification");
            writer.WriteStartElement("wp", "Text1", "WPNotification");
            writer.WriteValue(text1);
            writer.WriteEndElement();
            writer.WriteStartElement("wp", "Text2", "WPNotification");
            writer.WriteValue(text2);
            writer.WriteEndElement();
            writer.WriteEndElement();
            writer.WriteEndDocument();
            writer.Close();

            var payload =  Encoding.UTF8.GetString(stream.ToArray());
            DisplayString(payload); // Inspect bytes
            return payload;
        }

Conclusion

So there you have it. Once you know what the problem is – an unwanted BOM – it’s quite easy to find posts that discuss it and give you a solution. But many developers who are having difficulty with XML they have created in code will not know what the problem is they are trying to fix. No amount of studying the code, from inspecting strings in Visual Studio, or quite likely the exception message will tell you that maybe there is a hidden and unwanted character at the start of the generated XML. For you, hopefully this post will help you find a solution.

Beware the perils of async/await in application lifecycle event handlers (in fact in *any* event handlers)

Many developers are, quite rightly, adopting the use of async and await when writing Windows Phone and Windows 8 Store apps. It’s the right thing to do, as it allows long-running operations such as network calls and file i/o to complete efficiently without any risk of inadvertently locking up the UI thread and creating a poor user experience by freezing the screen responsiveness. In fact, in Windows 8 Store apps, it’s the *only* way to do these operations because in the .NET API for Windows Store apps, the old synchronous methods have been removed and only the new async versions are available. On Windows Phone 8, the developer has the choice of using the old Window Phone 7-style APIs, or in many cases newer async alternatives, such as the WinRT Windows.Storage APIs for file operations.

However, we’re seeing a lot of postings in the forums saying that their app only works intermittently, or works fine in debug but then fails intermittently when deployed. Quite often, the reason for this is that the developer has inadvertently called some async methods in a ‘fire and forget’ fashion, and what they are seeing is a synchronization problem; they assume that some async code has completed, and some code elsewhere is trying to use the result value from the async method before the async method has completed. The result, a null reference exception or incorrect behaviour because some object is not fully populated.

The problem with async/await in Application_Launching and Application_Activated

I accidently encountered this problem when I was building a simple demo for the last Windows Phone 8 JumpStart video series. OK, it was a dumb demo, but demos are often dumb because you want to keep them simple so as not to distract from the main point you are trying to make. This demo was to show how to program with the new Windows.Storage apis, so in my App class Application_Launching event handler I had code that read some data out of a file and saved the string into a public property called LogData:

    public partial class App : Application
    {
        ...

        // This is shared amongst all the pages
        // It is the contents of the log itself, as a large string
        public string LogData;

        // Code to execute when the application is launching (eg, from Start)
        // This code will not execute when the application is reactivated
        private async void Application_Launching(object sender, LaunchingEventArgs e)
        {            
            // Warning - this async method may not complete before this event handler method completes and other
            // events fire. Be careful not to assume completion of the logic in this method in other event handler code.
            this.LogData = await LoadFromLocalFolderAsync();
        }

        public async static Task<string> LoadFromLocalFolderAsync()
        {
            string theData = string.Empty;

            // There's no FileExists method in WinRT, so have to try to get a reference to it
            // and catch the exception instead
            StorageFile storageFile = null;
            bool fileExists = false;
            try
            {
                // See if file exists
                storageFile = await Windows.Storage.StorageFile.GetFileFromApplicationUriAsync(
                    new Uri("ms-appdata:///local/CaptainsLog.store"));
                fileExists = true;
            }
            catch (FileNotFoundException)
            {
                // File doesn't exist
                fileExists = false;
            }

            if (!fileExists)
            {
                // Initialise the return data
                theData = string.Empty;
            }
            else
            {
                // File does exist, so open it and read the contents
                Stream readStream = await storageFile.OpenStreamForReadAsync();
                using (StreamReader reader = new StreamReader(readStream))
                {
                    theData = await reader.ReadToEndAsync();
                }
            }

            return theData;
        }

And in MainPage.xaml.cs, OnNavigatedTo, the code gets the value of LogData from the App class and sets the Text property of a TextBlock to the string to show the contents of the file on the screen. Logic elsewhere in the app writes content into the file, so on startup you should always see the current contents of the file shown on the screen on application launch.

        protected override void  OnNavigatedTo(System.Windows.Navigation.NavigationEventArgs e)
        {
             base.OnNavigatedTo(e);
            
            // When we navigate to the page - put the log text into the display
            completeLogTextBlock.Text = ((App)App.Current).LogData;
        }

What could possibly go wrong? Well, plenty as it turns out! The first time you run this app, the screen is empty which is to be expected because the file is empty. But then you use the app to enter some text which is written to the file, then close the app. Launch again, and where you would expect to see some text displayed, the screen is still empty! What gives?

Exploring execution of Async methods

To show why this went wrong, let’s look at a simple console app to explore the two ways you can execute an async method.

using System;
using System.Threading.Tasks;

namespace AsyncSynchronizationProblem
{
    class Program
    {
        static void Main(string[] args)
        {
            new TheClassThatDoesWork().DoTheWork();

            Console.ReadKey();
        }
    }

    class TheClassThatDoesWork
    {
        public async void DoTheWork()
        {
            Console.WriteLine("TRACE Calling SomeLongProcedure Async");
            await SomeLongProcedureAsync();
            Console.WriteLine("TRACE Returned from SomeLongProcedure Async");

            Console.WriteLine();

            Console.WriteLine("TRACE Calling SomeLongProcedure Fire and forget");
            SomeLongProcedureAsync();
            Console.WriteLine("TRACE Returned from SomeLongProcedure Fire and forget");
        }

        private Task SomeLongProcedureAsync()
        {
            var task = Task.Run(() =>
                {
                    Console.WriteLine("TRACE Starting SomeLongProcedure work");
                    System.Threading.Thread.Sleep(1000); // Simulate a long running operation
                    Console.WriteLine("TRACE Completed SomeLongProcedure work");
                });

            return task;
        }
    }
}

When you run this app, you see the following console output:

asyncdemo1

In this sample, the long running method, Task SomeLongProcedureAsync() simply sleeps for 1s to simulate executing some lengthy code. In the method DoTheWork(), we just call the async method twice, once using await and once without. We write to the console output to show what is happening. I’m sure it is no surprise to most of you, that when we call using await, the caller waits until the long-running work has completed before continuing execution itself:

TRACE Calling SomeLongProcedure Async
TRACE Starting SomeLongProcedure work
TRACE Completed SomeLongProcedure work
TRACE Returned from SomeLongProcedure Async

and when we call without using await, it works in a fire-and-forget fashion, and the caller continues execution without waiting for the task to complete:

TRACE Calling SomeLongProcedure Fire and forget
TRACE Starting SomeLongProcedure work
TRACE Returned from SomeLongProcedure Fire and forget

TRACE Completed SomeLongProcedure work

This is pretty basic stuff about calling async methods. So why is this relevant?

Dangers of making event handlers async

Let’s look at another example. This one is still a console app, but has four classes, Program which just hosts the Main method of the console app, App, PhoneApplicationService and MainPage – I have named the last three the same as key classes in a Windows Phone app for reasons which hopefully will become obvious.

using System;
using System.Threading.Tasks;

namespace AsyncSynchronizationProblemWithEvents
{
    class Program
    {
        static void Main(string[] args)
        {
            var phoneAppService = new PhoneApplicationService();
            var app = new App(phoneAppService);
            
            phoneAppService.DoTheWork();

            // When the PhoneApplicationService has completed, construct a MainPage instance
            var mainPage = new MainPage();
            // Call methods on it
            mainPage.OnNavigatedTo();

            Console.ReadKey();
        }
    }

    class App
    {
        private PhoneApplicationService appService;

        public App(PhoneApplicationService applicationService)
        {
            appService = applicationService;
            // Subscribe to the PhoneApplicationService lifetime event(s)
            appService.ApplicationLaunching += Application_Launching;
        }

        async void Application_Launching(object sender, EventArgs e)
        {
            Console.WriteLine("TRACE Event handler - Calling SomeLongProcedure Async");
            await SomeLongProcedureAsync();
            Console.WriteLine("TRACE Event handler - Returned from SomeLongProcedure Async");
        }

        private Task SomeLongProcedureAsync()
        {
            var task = Task.Run(() =>
            {
                Console.WriteLine("TRACE Starting SomeLongProcedure work");
                System.Threading.Thread.Sleep(1000); // Simulate a long running operation
                Console.WriteLine("TRACE Completed SomeLongProcedure work");
            });

            return task;
        }
    }

    class PhoneApplicationService
    {
        public event EventHandler ApplicationLaunching;

        public void DoTheWork()
        {
            // Here we do some stuff...

            // Call any subscribers to our event(s)
            if (ApplicationLaunching != null)
            {
                ApplicationLaunching(this, new EventArgs());
            }
        }
    }

    class MainPage
    {
        public void OnNavigatedTo()
        {
            Console.WriteLine("TRACE Starting MainPage OnNavigatedTo");

            // Do some work here too...

            Console.WriteLine("TRACE Completed MainPage OnNavigatedTo");
        }
    }
}

If you study this app, you will see that the Main method creates instances of PhoneApplicationService, creates an instance of App and passes the PhoneApplicationService instance into its constructor, and finally creates an instance of MainPage and calls its OnNavigatedTo method. This is, of course, exactly what happens when you start a Windows Phone app. When you run it, the output looks like this:

asyncdemo2

There is clearly something odd going on here: the trace messages from the execution of SomeLongProcedure are output, but right in the middle, the calls to MainPage OnNavigatedTo appear. What’s going on?

Inside the App constructor, we hook the ApplicationLaunching event of the PhoneApplicationService instance so that in the event handler we can do some setup work when the app launches, and this of course is where we call the long-running async method:

        async void Application_Launching(object sender, EventArgs e)
        {
            Console.WriteLine("TRACE Event handler - Calling SomeLongProcedure Async");
            await SomeLongProcedureAsync();
            Console.WriteLine("TRACE Event handler - Returned from SomeLongProcedure Async");
        }

We are using the await keyword inside this event handler, so of course we mark it with the async modifier as normal. So at first glance, the developer expects this code to execute and for the event handler code to wait for completion of SomeLongProcedureAsync() before continuing execution, which in this case is simply to write a message to the console. If we look at the output, that is indeed what happens:

TRACE Event handler – Calling SomeLongProcedure Async
TRACE Starting SomeLongProcedure work

TRACE Starting MainPage OnNavigatedTo
TRACE Completed MainPage OnNavigatedTo

TRACE Completed SomeLongProcedure work
TRACE Event handler – Returned from SomeLongProcedure Async

But as you can see from the trace output, the MainPage OnNavigatedTo method is executed before the Application_Launching event handler has completed, even though it was clearly invoked by the Main function after the PhoneApplicationService.DoWork() method was invoked, which is the method that fires the Application_Launching event:

        static void Main(string[] args)
        {
            var phoneAppService = new PhoneApplicationService();
            var app = new App(phoneAppService);
            
            phoneAppService.DoTheWork();

            // When the PhoneApplicationService has completed, construct a MainPage instance
            var mainPage = new MainPage();
            // Call methods on it
            mainPage.OnNavigatedTo();

            Console.ReadKey();
        }

The problem of course, is that there is no way of awaiting the completion of an event handler, so when the DoWork() method fires the Application_Launching event, it does so in a fire and forget fashion, just the same as in the calling of SomeLongProcedureAsync() without using the await keyword, as we demonstrated in the first example. During execution, code executes sequentially until we hit the await keyword inside the Application_Launching event handler, at which point the wonders of the Task Parallel Library ensure that execution of the Task continues without blocking the original context and control returns immediately to the caller (PhoneApplicationService.DoTheWork() in this case). 

If the Application_Launching event handler had returned async Task instead of async void: 

async Task Application_Launching(object sender, EventArgs e)

(which is not possible with event handlers) and if it had been called using the await keyword, then  DoWork() would have awaited completion of the Application_Launching event handler logic before continuing, in which case, MainPage.OnNavigatedTo() would not have been called before the Application_Launching logic had completed. You can only await completion of methods that return Task or Task<T> and since this is impossible with event handlers, you can never await their completion. Hence there is every chance that your MainPage OnNavigatedTo logic will execute before your Application_Launching or Application_Activated logic has completed, although it is actually hit or miss, which is why so many developers report intermittent failures.

The Solution?

Of course, this is not a new problem. Any time you execute code asynchronously from your application lifecycle event handlers, whether using async/await or other asynchronous programming techniques, there is a real chance that you will have to implement some kind of synchronisation logic to ensure that your page logic does not try to access objects before your startup logic has set them up. Using older async coding technologies, this was perhaps a little more obvious to the developer, but many devs use the new Task-based async methods without thinking too deeply about it. It just works – or at least seems to until you hit a timing-related problem such as the one I’ve described here.

In the simple demo I described at the beginning of this article, the solution was simple. Instead of exposing the data fro the app in a public property of the App class, I created a new ViewModel class to host it and used XAML databinding to bind the TextBlock on MainPage to a property of the ViewModel. When the Application_Launching logic completed, it updated the ViewModel instance, and through the wonders of INotifyPropertyChanged, the MainPage UI updated with the correct data.

For other solutions, you may need to implement some other mechanism. One example was shown me by Peter Torr of the Windows Phone Product Group and is shown in the sample app which you can download from my post on SQLite:  SQLite-WinRT: Database programming on Windows Phone and Windows 8 . That sample opens (and the first time, creates) the SQLite database in Application_Launching and when resuming from tombstoning in Application_Activated. That sample used an old-fashioned synchronisation primitive, a ManualResetEvent, as a flag that is only set when the application lifecycle logic has finished:

  public partial class App : Application
  {
    static SQLiteWinRTPhone.Database db;
    static ManualResetEvent DBLoaded = new ManualResetEvent(false);

    public static Task<SQLiteWinRTPhone.Database> GetDatabaseAsync()
    {
      return Task.Run(() =>
      {
        DBLoaded.WaitOne();
        return db;
      });
    }

    // Code to execute when the application is launching (eg, from Start)
    // This code will not execute when the application is reactivated
    private void Application_Launching(object sender, LaunchingEventArgs e)
    {
      LoadDatabase();
    }

    private async Task LoadDatabase()
    {
      // Get a reference to the SQLite database
      db = new SQLiteWinRTPhone.Database(ApplicationData.Current.LocalFolder, "sqlitedemo.db");

      await db.OpenAsync();

      string sql = @"CREATE TABLE IF NOT EXISTS
                                Customer (Id      INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
                                          Name    VARCHAR( 140 ),
                                          City    VARCHAR( 140 ),
                                          Contact VARCHAR( 140 ) 
                            );";
      string description = "Create Customer table";
      await ExecuteSQLStatement(db, sql, description);

      sql = @"CREATE TABLE IF NOT EXISTS
                                Project (Id          INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
                                         CustomerId  INTEGER,
                                         Name        VARCHAR( 140 ),
                                         Description VARCHAR( 140 ),
                                         DueDate     DATETIME,
                                         FOREIGN KEY(CustomerId) REFERENCES Customer(Id) ON DELETE CASCADE 
                            )";
      description = "Create Project table";
      await ExecuteSQLStatement(db, sql, description);

      // Turn on Foreign Key constraints
      sql = @"PRAGMA foreign_keys = ON";
      description = "Enable foreign key constraints";
      await ExecuteSQLStatement(db, sql, description);

      DBLoaded.Set();
    }

    private static async Task ExecuteSQLStatement(SQLiteWinRTPhone.Database db, string sql, string description)
    {
      try
      {
        await db.ExecuteStatementAsync(sql);
        Debug.WriteLine(description + " executed OK");
      }
      catch (Exception ex)
      {
        var result = SQLiteWinRTPhone.Database.GetSqliteErrorCode(ex.HResult);
        throw new ApplicationException(description + " Failed with error " + result);
      }
    }

The ManualResetEvent is declared unsignalled as a static field of the app class:

 static ManualResetEvent DBLoaded = new ManualResetEvent(false);

and is only set at the very end of the async Task LoadDatabase() method:

private async Task LoadDatabase()
{
  // Get a reference to the SQLite database
  db = new SQLiteWinRTPhone.Database(…);
 
await db.OpenAsync();
  …

  DBLoaded.Set();
}

The final key ingredient is a ‘gatekeeper’ method which must be called by every other method in the app that wants to access the database:

public static Task<SQLiteWinRTPhone.Database> GetDatabaseAsync()
{
  return Task.Run(() =>
  {
    DBLoaded.WaitOne();
    return db;
  });
}

This method waits for the DBLoaded semaphore to be set and does so on a Threadpool thread (through Task.Run()) so does not block the UI thread.

Calling async code from Application_Deactivated or Application_Closing

The guidance here is “don’t’”. If you write your apps carefully, you can be saving changes to persistent data as you go along, so you shouldn’t have anything to do in the application lifecycle events.

If you must, you can try doing something like this:

SomeAsyncMethod().AsTask().Wait()   

If the operation completes within the timeout period AND it doesn’t deadlock due to needing to pump the UI thread, it will work… but don’t count on it.

SQLite-WinRT: Database programming on Windows Phone and Windows 8

Today at TechEd North America in New Orleans, I am presenting a session on programming the SQLite database on Windows Phone 8. You can watch the video and download the slides from here: Support Your Demanding LOB Apps With SQLite and Windows Phone 8. The sample application I walked through is here:

[Updated 21-Oct-2013 – SQLite 3.8.2]

The session is a deep dive on the new SQLite-WinRT API that we released on Codeplex last week. This new API is a thin wrapper around the native database engine, sqlite3.dll and exposes the key methods of the database engine through a modern Windows Runtime API callable from C#, Visual Basic and C++ that is all async methods and which reports unexpected results by throwing exceptions. The code is all up at http://sqlwinrt.codeplex.com. There’s no download for this project – you just download the source code, which is a complete Visual Studio 2012 solution containing projects for Windows Phone 8 and Windows 8 Store App components.

Why another database API on Windows Phone?

That’s a valid question! After all, we already have the Local Database in the Windows Phone SDK which is a LINQ to SQL API where the underlying database is SQL Server Compact Edition 3.7-ish. And we also have the SQLite-NET API for accessing SQLite which also exposes a LINQ API for working with a SQLite database. The Local Database LINQ to SQL API is not available for Windows 8 Store App development so not a great choice if you want to share code between your phone and tablet/PC apps, but SQLite-NET is available for both phone and Windows 8, as is SQLite-WinRT – so why do we need this new API?

SQLite-WinRT is for those developers who prefer to work with SQL statements rather than the object-oriented layer that LINQ gives you. It is also for those coming to Windows Phone 8 or Windows 8 from other platforms who have worked with SQLite before probably programming with C/C++ – the methods exposed by the SQLite-WinRT API are very thin wrappers around the SQLite C/C++ API. It’s all about giving choice to our developers – we have heard feedback that not all developers like to work with a local database using LINQ, so this API is for them.

To give you a comparison, this is how you might select a Customer from the Customer table using SQLite-NET:

var db = new SQLite.SQLiteAsyncConnection(App.DBPath); 
var _customer = await (from c in db.Table<Customer>() 
                        where c.Id == customerId 
                        select c).FirstOrDefaultAsync(); 
if (customer != null) 
{ 
    var Id = _customer.Id; 
    var Name = _customer.Name; 
}

And this is how you would do the same thing in SQLite-WinRT:

using (var db = new SQLiteWinRT.Database(
    ApplicationData.Current.LocalFolder, "demo.db")) 
{ 
    await db.OpenAsync(); 
    using (var stmt = await db.PrepareStatementAsync(
                        "SELECT name, age FROM people")) 
    { 
        while (await stmt.StepAsync()) 
        { 
            var name = stmt.GetTextAt(0); 
            var age = stmt.GetIntAt(1); 
        } 
    } 
}

The choice is yours!

Getting Started with SQLite-WinRT

Start by installing the SQLite Extension SDK. This is the native dll, sqlite3.dll, which contains the database engine. To install this, in Visual Studio, on the Tools menu, open Extensions and Updates. In the left hand pane, select Online – Visual Studio Gallery, then enter SQLite into the search box. Select SQLite for Windows Phone and click the Download button to download and install the vsix package. (The same procedure works for the SQLite for Windows Runtime extension for Windows 8 Store Apps.)

image_thumb3

After you’ve installed that, the native sqlite3 dlls are installed into a folder under C:Program Files (x86)Microsoft SDKsWindows Phonev8.0ExtensionSDKsSQLite.WP80. That’s just for your information – you should never have to manually copy the sqlite3.dll from there in order to use the database

Warning: You will be prompted whenever an update is available for the SQLite database engine. When you install it, you will inadvertently ‘break’ SQLite-WinRT, in the sense that Visual Studio will be unable to open the project. That is because the project references the C:Program Files (x86)Microsoft SDKsWindows Phonev8.0ExtensionSDKsSQLite.WP803.7.17 path in a number of places, and when you upgrade SQLite, the version part of that path will change.
I will be adding a powershell script into the download to make it easy to update the SQLite-WinRT project file with the correct path, but if this happens to you before I do it, just edit the SQLiteWinRTPhone.vcxproj file (carefully!) to correct references to the old version and update them to pojnt at whatever new version you have installed.

Creating your Project

Create your C# or Visual Basic project as normal. Then go to http://sqlwinrt.codeplex.com and download the source code from there – remember there is no ‘official download’ for this Codeplex project.

image_thumb7

The source code for this project is a solution containing two projects, the SQLite-WinRT component for Windows Phone, and the same thing but for Windows 8 Store Apps. Copy the Windows Phone project and use ‘Add Existing Project…’ to add it into your solution, then add a reference to the SQLite-WinRT project from your C#/VB project. That’s all you have to do!

Remember, if Visual Studio cannot load the SQLite-WinRT project, it’s probably because the SQLite extension has been updated. Read the Warning a few paragraphs before to resolve this.

Creating a database

To create a database, you can use code such as this:

private void Application_Launching(object sender, LaunchingEventArgs e)
    {
      LoadDatabase();
    }

    private async void LoadDatabase()
    {
      // Get a reference to the SQLite database
      db = new SQLiteWinRT.Database(
          ApplicationData.Current.LocalFolder, "sqlitedemo.db");

      await db.OpenAsync();

      string sql = @"CREATE TABLE IF NOT EXISTS
                    Customer (Id      INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
                                Name    VARCHAR( 140 ),
                                City    VARCHAR( 140 ),
                                Contact VARCHAR( 140 ) 
                );";
      string description = "Create Customer table";
      await ExecuteSQLStatement(db, sql, description);

      sql = @"CREATE TABLE IF NOT EXISTS
                    Project (Id          INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
                                CustomerId  INTEGER,
                                Name        VARCHAR( 140 ),
                                Description VARCHAR( 140 ),
                                DueDate     DATETIME,
                                FOREIGN KEY(CustomerId) REFERENCES Customer(Id) 
                                    ON DELETE CASCADE 
                )";
      description = "Create Project table";
      await ExecuteSQLStatement(db, sql, description);

      // Turn on Foreign Key constraints
      sql = @"PRAGMA foreign_keys = ON";
      description = "Enable foreign key constraints";
      await ExecuteSQLStatement(db, sql, description);

      DBLoaded.Set();
    }

    private static async Task ExecuteSQLStatement(
        SQLiteWinRT.Database db, 
        string sql, 
        string description)
    {
      try
      {
        await db.ExecuteStatementAsync(sql);
        Debug.WriteLine(description + " executed OK");
      }
      catch (Exception ex)
      {
        var result = 
            SQLiteWinRTPhone.Database.GetSqliteErrorCode(ex.HResult);
        throw new ApplicationException(description + 
            " Failed with error " + result);
      }
    }

You can call this code from your Application_Loaded and Application_Activated event handlers. One particular thing I want to call out here: notice the last line of the LoadDatabase method:

DBLoaded.Set();

DBLoaded is a ManualResetEvent which is used to flag when the database opening logic has completed. It is declared at the top of my App.Xaml.cs class, along with a ‘Gatekeeper’ method called GetDatabaseAsync():

static SQLiteWinRT.Database db;
    static ManualResetEvent DBLoaded = new ManualResetEvent(false);

    public static Task<SQLiteWinRT.Database> GetDatabaseAsync()
    {
      return Task.Run(() =>
      {
        DBLoaded.WaitOne();
        return db;
      });
    }

The ManualResetEvent is created unset, and doesn’t get set until the last line of LoadDatabase(). All other database access code in my viewmodel classes makes a call to GetDatabaseAsync as the first thing it does. The DBLoaded.WaitOne() call in there blocks until DBLoaded is set, so this prevents any other database accessing code from executing until the LoadDatabase() method has completed.

Handling Errors

The SQLWinRT API throws exceptions when something goes wrong. Unfortunately, exceptions thrown in a WinRT component are not the most developer-friendly things ever devised. They are COMException objects, have no useful error text in them and just return to you a HResult. But you can get a meaningful error message if you call the handy GetSqliteErrorCode method which is used like this:

try
{
await db.ExecuteStatementAsync(sql);
}
catch (Exception ex)
{
var result = 
    SQLiteWinRT.Database.GetSqliteErrorCode(ex.HResult);
throw new ApplicationException("Failed with error " + result);
}

GetSqliteErrorCode() returns the – yes, you guessed it – SQLite error code, which mostly are self explanatory (such as ‘SQLITE_CONSTRAINT’) are documented at http://www.sqlite.org/c3ref/c_abort.html.

CRUD operations

Create

To create records in the database, use code similar to this:

try 
{
    // Connection already opened in app.xaml.cs - get reference     SQLiteWinRT.Database db = App.db;     using (var custstmt = await db.PrepareStatementAsync(
    "INSERT INTO Customer (Name, City, Contact) VALUES (@name, @city, @contact)"
      ))
    {         // NOTE that named parameters have a leading "@",":" or "$".         custstmt.BindTextParameterWithName("@name", customer.Name);
        custstmt.BindTextParameterWithName("@city", customer.City);         custstmt.BindTextParameterWithName("@contact", customer.Contact);
        // Use StepAsync to execute a prepared statement         await custstmt.StepAsync();     } 
} 
catch (Exception) 
{ … }

This uses a SQL statement containing named parameters (@name, @city, @contact), which you prepare using PrepareStatementAsync() and which you ‘plug in’ the values using the BindtypeParameterWithName(string parametername, type value) method. An alternative is to use anonymous parameters (shown below when describing Update).

Notice that we use the StepAsync() method to execute a row-returning SQL statement. StepAsync may seem curious, but the SQLite C/C++ API method this wraps is called Step(), so that’s the name we adopt for our WinRT method.

IMPORTANT: Remember to dispose of your Statement objects when you’re done with them – a good way of doing that is to use the C# using statement, as shown above.

Read

To read objects, use a SELECT statement:

public async Task<CustomerViewModel> GetCustomerAsync(int customerId)
{
    CustomerViewModel customer = null;
    using (var readstmt = await App.db.PrepareStatementAsync(
                "SELECT Id, Name, City, Contact FROM Customer WHERE Id = " 
                + customerId))
    {
        if (await readstmt.StepAsync() == true)
        {
            var customer = new CustomerViewModel()
            {
                Id = readstmt.GetIntAt(0),
                Name = readstmt.GetTextAt(1),
                City = readstmt.GetTextAt(2),
                Contact = readstmt.GetTextAt(3)
            };
        }
    }
    return customers;
}

StepAsync() returns true if there is a row to return. The example above would reasonably be expected to return just a single row, but if you get multiple rows returned from the database, then you would loop around creating your in-memory objects (your viewmodel objects) and entering them into some collection until StepAsync() returns false.

An alternative form of SELECT allows you to extract data from the rowset returned from the database by using the column name, rather than the column position as the previous code sample did. You enable this behaviour by calling the Statement.EnableColumnsProperty() method. Then the results are returned as a Dictionary<string, string> which you key into using the column name:

using (var readstmt = await App.db.PrepareStatementAsync(
    "SELECT Id, Name, City, Contact FROM Customer WHERE Id = " 
    + customerId))
{
    // Enable the columns property
    statement.EnableColumnsProperty();

    if (await readstmt.StepAsync() == true)
    {
        // it isn't super useful as all columns are returned as text and must be parsed 
        var columns = statement.Columns;
        var customer = new CustomerViewModel()
        {
            Id = int.Parse(columns["Id"]),
            Name = columns["Name"],
            City = columns["City"],
            Contact = columns["Contact"]
        };
    }
}  

This does impose a slight performance overhead, and every value returns as a string, so you have to parse the string to get the correct value type, as shown here for the Id column.

Update

Update is simple enough. This sample uses anonymous parameters in the SQL statement, rather then the named parameters we saw when discussing the Insert operation. Note that anonymous parameters are 1-based, not 0-based like just about every other collection in .NET!

// See if the customer already exists
var existingCustomer = await GetCustomerAsync(customer.Id);
if (existingCustomer != null)
{
    using (var custstmt = await App.db.PrepareStatementAsync(
        "UPDATE Customer SET Name = ?, City = ?, Contact = ? WHERE Id=?"))
    {
        // NOTE when using anonymous parameters the first has an index of 1, not 0. 
        custstmt.BindTextParameterAt(1, customer.Name);
        custstmt.BindTextParameterAt(2, customer.City);
        custstmt.BindTextParameterAt(3, customer.Contact);
        custstmt.BindIntParameterAt(4, customer.Id);

        await custstmt.StepAsync();
    }
}

Delete

Delete is simple enough:

string sql = @"DELETE FROM Customer WHERE Id={0}";
sql = string.Format(sql, customerId);

// Can use ExecuteStatementAsync to run non row returning statements
await App.db.ExecuteStatementAsync(sql);

Summary

That’s an introduction to getting started with SQLite-WinRT. In the next posts, I will talk some more about Foreign Key constraints and about working with Read-only databases.