Baby steps into the Internet of Things #1

First Steps #1: What is the Internet of Things?

You might have noticed a bit of chatter recently about something called the “Internet of Things” or IoT for short. It’s a classical industry hype buzz – everyone talking about it, a lot of people worried that they’re going to miss out on this ‘next big thing’ if they’re not careful. In fact, Gartner has called it as currently right at the peak of  the curve of ‘hype, and unrealistic expectations’ on their classical curve of technology adoption – which means the inevitable ‘trough of disillusionment’ is only just around the corner!

But this thing is real, and it’s exciting. I can make an honest claim to have been in IoT for many years, since I have been working for many years writing apps with the .NET Compact Framework on small processors running Windows CE in embedded applications, and on small handheld computers running Windows Mobile or Windows CE and more recently Windows Phone.

But all that is a niche (but very interesting) area using relatively pricey computers, and the real reason for the excitement around IoT is that we can finally start to deliver on having really cheap computers and sensors incorporated into ‘things’ around us, delivering data that we can use to create really interesting solutions. And I want to be in on this too, so I’ve started to explore it a bit and I’m going to document my ‘journey’ in a short series of blog posts – this is the first.

Enter ‘The Makers’

So I get that in any IoT solution, there are ‘things’ which are gathering data. These ‘things’ can be very simple, such as PIR movement detectors, light sensors, temperature sensors or GPS locators, or more sophisticated such as NFC readers or facial recognition solutions to detect when a specific person has entered a location.

SesnorsEverywhere

These smart things can be modelled and tested out using development boards where you can prototype this basic information gathering. There are a number of cheap electronics development boards out there, such as Arduino, Raspberry Pi, Netduino, Intel Galileo to mention just a few, which electronics-savvy people can wire up to simple stuff such as sensors, LEDs, visual displays, buzzers or sounders, or to more sophisticated devices such as motors and actuators to do cool stuff. And these boards can easily be networked to the internet or to other things where they can send the data that they gather. The people who work with these development boards are collectively known as ‘makers’ and there’s a lively community out there, all helping each other to make cool stuff, and readily giving help to beginners.

For me – this is perfect! Way back in the mists of time, I studied at Manchester University and my degree course was Electronics. But on my course, I realised I was more interested in this relatively new industry called Computer Programming than I was in soldering, so after I got my degree, I’ve been working in software ever since and hardly done anything with electronics. Which seems a bit of a shame – so now I can go back to my roots, so to speak, and combine both a bit of electronics with my love of software solutions and hopefully build something exciting!

If you have no interest in electronics, don’t worry. You don’t have to know how to use a soldering iron to get involved with the IoT. This thing will be huge, and the cheap electronics that are embedded in the ‘things’ is just one part of it. Sure prototyping electronics boards is one part of the whole process, but once the prototype sensor bit of it has been perfected using a development board, then some factory somewhere will manufacture the production version of it which will end up built into your smart refrigerator, your entry door, your environmental sensor – or whatever it is. Then the data starts flowing and it’s time to build a solution that makes use of that data.

So, what does an IoT solution look like?

This was the first question I asked, as a beginner in this field. I had an idea that there was small electronics boards in there. The classical example is the smart refrigerator, which warns you when your milk has been in the fridge too many days so you can throw it out, or which automatically adds items to your electronic shopping list as you use them up. OK – but how would that actually work?

How would a smart refrigerator actually work?

Well, that kind of scenario isn’t easy. For starters, the smart refrigerator would have to ‘know’ what items were in your fridge. I guess you could try to incorporate a barcode scanner in the door so that each time an item goes in and out, the user would have to scan the code on the packaging. Hmmm – no, that’s no good: too intrusive for the user. We could get an industry consensus-thing going so that every food manufacturer put an NFC tag on their packaging! Yeah, great idea, not going to happen.

So today, you’d have to sell your smart refrigerator with a big bag of smart tags, which the user would attach to products when they bought them, and which would be separated into different categories, maybe, milk, butter and any other specific products that the user wants to track. That’s probably OK since the user chooses which products to track, they stay in control and probably the only annoyance would be when the user accidentally throws out another smart tag with the trash and periodically has to purchase some more? Although – actually these pretty dumb tags could be passive NFC and maybe they would be cheap enough to throw away, and maybe throwing them away would be a good thing, since that might avoid mistakes like your fridge telling you your milk has gone off when actually all you’ve done is to re-use a tag but attached it to a new milk carton.

Alright, so now we’ve tracked the products and now the fridge can gather data about products going in and out of your fridge. What should it do with that data?

What to do with all that data?

This is where you start to see the anatomy of a complete IoT solution appear. The fridge itself could have software running in it to make decisions based on the data that it has – but then it would also have to be updated when software updates become available. And it would probably also have to offer some kind of UI so that the end user can configure their fridge. This doesn’t sound like a flexible, extensible solution.

So wouldn’t it be better for the fridge to simply take all that data and transmit it up to a database in the cloud, where the smart stuff would happen? Jobs would run in the cloud to analyse the stream of event data coming from smart refrigerators. The cloud would also host the end-user website where they can configure their fridge and determine what kind of alerts should be raised and what actions to take when their milk has been in the fridge for four days, and because it is in the cloud, it can easily be updated with new features.

The fridge itself acts as a gateway to ship the raw data up to the cloud, but it also acts as a consumer of the outputs of the processing and has buzzers and a display panel to alert the user to messages which have come down from the cloud as a result of the analysis of the raw data. But think how easy it would be to also send alerts to the users’ smartphone, or email, or SMS or…?

IoT-simplified

This sounds good! Now we’ve got a good solution for this. Good for the end-user – but also good for the manufacturer of the smart fridge who not only offers a good service to consumers, but can also get diagnostic messages from their smart refrigerators through the cloud, and also gets some potentially valuable usage data about what users are putting in their fridges which could guide their future product development and could possibly be sold onto other organisations– with end user permission of course.

The basic anatomy of an IoT solution

So with that I think I’ve figured out the basic layout of an IoT solution: sensors gather data which you ship to the cloud where you process it and publish the results to some consumer(s) – which could be a website, a smartphone or computer, or some other device.

Next thing for me is to take a closer look at the data gathering bit of all this and to get my hands dirty with a development board. I’m going to get myself a development board and have my first experiences of being a ‘maker’! That’ll all be in my next post!

Building Windows Phone, Windows and Universal Apps: Winter Events Series!

*Update*: Apologies, we’ve had to cancel the Dev Workshop on Thursday 20th November.

In the late summer, we held a series of events aimed at Windows Phone and Windows 8.1 app developers intended to help you along the road to getting an app published in the Windows Store, whether you are a complete beginner or a more experienced app developer.

We are running a new series of those events this winter. We start with two one-hour webinars run back-to-back, follow those with a one day workshop at Microsoft’s offices in Victoria, London, and wrap up with a ‘Publish to the Store’ hack event again at Microsoft London:

Webinar 1: Windows Apps JumpStart LIVE Series – Kick Starter Webinar for the non C#/XAML developer: This one hour webinar is aimed at developers who are completely new to Visual Studio, XAML, C# and .NET. This will give you what they need to get started with downloading Visual Studio, a little bit of knowledge of C# and the structure of a XAML-based app.
Online webinar
Friday 7 Nov 2014  10:00 –11:00
Registration: http://aka.ms/winappswebinar1

Webinar 2: Windows Apps JumpStart LIVE Series – JumpStart Webinar: Building Universal Apps with C# and XAML: This one hour webinar follows on straight away after the previous one. It is aimed at developers who are new to Windows Store app development, but who have some experience of developing software using .NET, or for those who have attended the previous webinar.
Online webinar
Friday 7 Nov 2014    11:00 – 12:00
Registration: http://aka.ms/winappswebinar2

Following on from the webinars, we have a one-day dev camp:

Dev camp: Windows Apps JumpStart LIVE Series: Windows Apps Dev Workshop
This one day event will be help at Microsoft’s offices near Victoria Station, London, running 9:30 – 17:30. At this workshop, you will learn how to program Windows and Windows Phone apps, including universal apps that run on both. There will be Hands-on Labs available, and you can get tips and advice from Microsoft experts and from peers in the developer community. There’ll be some giveaways as well Smile
Dev camp
Sorry! This one has had to be cancelled…

And following on from that, we have a one-day Publish hackathon:

Publish event: Windows Apps JumpStart LIVE Series: Publish Hackathon
This informal one-day event will also be held at Microsoft’s building in London, running 9:00 – 21:00 on a Saturday. This is a fun 12 hour event encouraging developers to bring their existing projects (apps/cloud projects) to polish and publish through to the last mile with help from Microsoft and community experts. There’ll be pizza and drinks and prizes too!
Publish Hack event
Saturday 13 December 2014
    09:00 – 21:00
Registration: http://aka.ms/winappspublishhack

Come along and join us – and if you don’t manage to publish an app this quarter, remember we are repeating these events again the first half of next year so you can get started now and then come and join us again later!

Windows Phone 8.1 Smart card Reader sample code

I have had quite a few queries from developers who watched the Building Apps for Windows Phone 9.1 Jump Start videos and are interested in the low-level smart card reading that I showed in module 15: Sensors and Proximity: NFC and Bluetooth in which I demoed an app that read data off a bank debit card. You can find all the samples we showed in the Jump Start at http://aka.ms/wp81samples but the smartcard sample is missing from there.

There were two reasons for that: firstly, there have been no phones available that can run it! Until now that is. Low-level card reading needs a new NFC chipset that is only available in the newest phones that are only now coming to market, such as the Lumia 730 and Lumia 830. The device I used in the video was an engineering prototype. And the second reason for not publishing it was that at the time we published the code samples, I wasn’t sure we had the go-ahead from the card company whose card we used.

Anyway, now the phones are becoming available and the NFC Smart card team at Microsoft have published their ‘official’ sample up on MSDN code samples: NFC Smart Card Reader PC/SC Library . The Home page of the project explains it all perfectly, so I’ll just repeat a shortened version here:

“Example code and library for raw communication to NFC tags with Windows Phone 8.1, but requires a device with support (Lumia 830 & 73x)
NOTE: This code will work only on a Windows Phone running 8.1 or higher, and with an NFC driver that supports smart card reader mode such as the NXP PN547 which is available on the Lumia 730 and Lumia 830 (it will NOT work on a Lumia 930 or any existing hardware, even after updating to 8.1). See inside the MainPage.GetDevices() function for how to detect at runtime whether a device supports this functionality.

A new feature in Windows Phone 8.1 (on specific new hardware devices) is to be able to directly send raw commands to external plastic NFC cards, previously the phone could only read/write NDEF messages to NDEF-formatted tags but now you can talk to things like transit passes etc. For NXP PN547-based devices NFC cards such as MIFARE Classic/Ultralight/DESfire, ISO7816-4/APDU-based cards like JCOP, Jewel are supported as well as some limited support for ISO15693 and Felica. This project contains sample code only for MIFARE Classic/Ultralight/DESfire and Felica, but can be easily extended to support the others.”

So there you go! Go download this sample, get yourself one of the new phones, and start building interesting apps that interact with transit passes, security passes and take advantage of other smart card use scenarios!

Hey, Cortana – How can I integrate my app into you?

At the recent IFA 2014 show in Berlin, Microsoft announced some great new phones, the budget Lumia 730 and 735 and the ‘affordable premium’ phone the Lumia 830. Exciting new hardware – but the standout new feature for many was the announcement of ‘Hey, Cortana’, a new accessibility feature for the popular voice-controlled assistant which allows you to launch the app just by saying ‘Hey, Cortana’ at any time, avoiding the need to press the search button or the Live Tile as you do currently. This feature will be available on the high-end devices with the Snapdragon 800 processor, such as the Lumia 930, 1520 and Icon, and will be shipped as part of the Lumia ‘Denim’ firmware update coming to phones in Q4 2014. Cortana has been getting many plaudits – rightly so – and is currently available in the US, on beta in the UK and China, and will soon be rolling out as a beta soon to Australia, India and Canada, with more countries next year.

clip_image002

Speech control is part of the next wave of improvements of how we interact with our devices, part of a loose grouping of technologies often called NUI (Natural User Interface), meaning human to device interface methods that are effectively invisible, i.e. ‘natural’, as distinct from artificial ways of interacting with a device such as pressing a button which have to be learnt.

So having Cortana capable of passive listening so you can just talk to it whenever the fancy takes you and wake it up is another small step towards making speech interaction with our devices easier and more natural. It removes one more piece of friction – the need to tap on a button or tile to start the voice interaction – that could prevent users choosing voice as their favoured means of interaction with their devices rather than one only used by enthusiasts.

Integrating Apps into Cortana

So that’s great, now we can say ‘Hey, Cortana’ to invoke the built-in digital assistant features of Cortana and get it to tell us upcoming events in our calendar, set new reminders, do searches etc. But what if you want to allow users to interact with your app in a similar way, use Cortana as a conduit for speech interaction with your app? Say something like: “Hey Cortana” to invoke Cortana, and then say “MSDN find ‘Windows Phone Voice Commands’” to invoke the MSDN Voice Search app.

Well, that’s not too hard! In fact, we’ve had the technology even in Windows Phone 8.0, before Cortana was a twinkle in Halo’s eye. Back in the Windows Phone 8.0 Jump Start videos, we recorded one session all about voice commands (http://channel9.msdn.com/Series/Building-Apps-for-Windows-Phone-8-Jump-Start/Building-Apps-for-Windows-Phone-8-Jump-Start-13-Speech-Input-in-Windows-Phone-8 ) which explained how to create a Voice Command definition file to allow you to launch apps with speech. It’s that same technology that is used to launch apps through speech from Cortana.

Although it was quite easy to implement, not that many developers added voice command activation to their 8.0 apps and very few users turned to voice as their favoured method of launching their favourite app. Why not? Discoverability! To activate voice recognition on a Windows Phone 8.0 device, you had to press and hold the Windows button. Most users only ever discovered this by accident, if at all! So as a consequence, it never gained wide usage.

That’s why Cortana is so cool – everyone knows that it is the voice portal on a Windows Phone app. It’s no longer down to knowing about some unnatural user interaction mechanism – a tap and hold on the Windows key – but now you invoke Cortana by a single tap on the Search button, or by tapping on the Live Tile, or even by launching it from the Apps list. Or – better by far – by simply saying ‘Hey, Cortana’ on a top-end phone with the new Lumia Denim software. [Note that in countries that do not have Cortana yet, you can still launch apps by voice command

Voice Command Recognition in Windows Phone 8.0

The Voice Commands implementation in Windows Phone 8.0 had another limitation apart from discoverability: it was a little too inflexible. With Windows Phone 8.0 Voice Commands, the full text of the commands themselves needed to be known ahead of time—this made them difficult to use for searching, social networking, or anything else with a large vocabulary.

For example, a Windows Phone 8.0 voice command definition looks like this:

    <CommandSet xml:lang="en-gb" Name="UKenglishCommands">
        <!-- The CommandPrefix provides an alternative to your full app name for invocation -->
        <CommandPrefix> MSDN </CommandPrefix>
        <!-- The CommandSet Example appears in the global help alongside your app name -->
        <Example> search </Example>

        <Command Name="MSDNSearch">
            <!-- The Command example appears in the drill-down help page for your app -->
            <Example> do a search </Example>

            <!-- ListenFor elements provide ways to say the command, including references to 
            {PhraseLists} and {PhraseTopics} as well as [optional] words -->
            <ListenFor> [do a] Search [for] {*} </ListenFor>
            <ListenFor> Find {*} </ListenFor>
            <ListenFor> Look [for] {*} </ListenFor>

            <!--Feedback provides the displayed and spoken text when your command is triggered -->
            <Feedback> Searching MSDN... </Feedback>

            <!-- Navigate specifies the desired page or invocation destination for the Command-->
            <Navigate Target="MainPage.xaml" />
        </Command>
    </CommandSet>

Every command must start with the CommandPrefix – ‘MSDN’ in this case. Thereafter, the ListenFor elements define the different variants for a specific speech command, and these may include optional words if they are enclosed in square brackets. There is the ability to allow for free speech, which is represented by {*} but that still had to be preceded by some fixed speech components in the command. So for this example, all of the following will trigger the voice command:

  • Search xyz
  • Search for xyz
  • Find xyz
  • Look xyz
  • Look for xyz

But the following will *not* be recognised because it doesn’t fit into one of the ListenFor definitions:

  • I want to know about xyz

Recognizing natural language in Windows Phone 8.1

The Voice Command Definition for 8.1 allows more flexibility:

<?xml version="1.0" encoding="utf-8"?>

<!-- Be sure to use the new v1.1 namespace to utilize the new PhraseTopic feature -->
<VoiceCommands xmlns="http://schemas.microsoft.com/voicecommands/1.1">
    <!-- The CommandSet Name is used to programmatically access the CommandSet -->
    <CommandSet xml:lang="en-gb" Name="UKenglishCommands">
        <!-- The CommandPrefix provides an alternative to your full app name for invocation -->
        <CommandPrefix>MSDN</CommandPrefix>
        <!-- The CommandSet Example appears in the global help alongside your app name -->
        <Example> find 'Windows Phone Voice Commands' </Example>

        <Command Name="MSDNSearch">
            <!-- The Command example appears in the drill-down help page for your app -->
            <Example> find 'how to install CommandSets' </Example>

            <!-- ListenFor elements provide ways to say the command, including references to 
            {PhraseLists} and {PhraseTopics} as well as [optional] words -->
            <ListenFor> Search </ListenFor>
            <ListenFor> Search [for] {dictatedSearchTerms} </ListenFor>
            <ListenFor> Find {dictatedSearchTerms} </ListenFor>
            <ListenFor> Find </ListenFor>

          <!--Feedback provides the displayed and spoken text when your command is triggered -->
            <Feedback> Searching MSDN... </Feedback>

            <!-- Navigate specifies the desired page or invocation destination for the Command-->
            <Navigate Target="MainPage.xaml" />
        </Command>

        <Command Name="MSDNNaturalLanguage">
            <Example> I want to go to the Windows Phone Dev center </Example>
            <ListenFor> {naturalLanguage} </ListenFor>
            <Feedback> Starting MSDN... </Feedback>
            <Navigate Target="MainPage.xaml" />
        </Command>

        <PhraseTopic Label="dictatedSearchTerms" Scenario="Search">
            <Subject> MSDN </Subject>
        </PhraseTopic>

        <PhraseTopic Label="naturalLanguage" Scenario="Natural Language">
            <Subject> MSDN </Subject>
        </PhraseTopic>

    </CommandSet>
</VoiceCommands>

Notice the following ListenFor elements:

        <Command Name="MSDNSearch">
           …
            <ListenFor> Search [for] {dictatedSearchTerms} </ListenFor>
           …
        </Command>
        <Command Name="MSDNNaturalLanguage">
            …
            <ListenFor> {naturalLanguage} </ListenFor>
            …
        </Command>

Both of these elements contain a label inside curly brackets which refers to the PhraseTopic elements defined towards the bottom. First dictatedSearchTerms is defined as:

        <PhraseTopic Label="dictatedSearchTerms" Scenario="Search">
            <Subject> MSDN </Subject>
        </PhraseTopic>

A PhraseTopic is a new element in 8.1 which specifies a topic for large vocabulary definition. What this means is that the voice input goes up to the cloud where it is recognized against the same large vocabulary of words that Cortana uses for its built-in functions. The Scenario attribute is used to guide the recognition. Valid values are “Natural Language”, “Search”, “Short Message”, “Dictation” (the default), “Commands”, and “Form Filling”. The Subject child element specifies a subject specific to the parent PhraseTopic’s Scenario attribute to further refine the relevance of speech recognition results.

So the command that uses this ListenFor definition allows free speech searches in MSDN.

The second ListenFor element has no fixed command terms at all:

<ListenFor> {naturalLanguage} </ListenFor>

Where naturalLanguage is defined as:

        <PhraseTopic Label="naturalLanguage" Scenario="Natural Language">
            <Subject> MSDN </Subject>
        </PhraseTopic>

Effectively, this defines an ‘anything goes’ command, so this can recognize a query such as “I want to know about xyz”. Consider this a free-form alternative to the other, more rigid, command labelled MSDNSearch where the ListenFor elements define fixed components of the speech command (‘Find’, ‘Search’) that must be present for recognition to succeed.

But how then does the app know what it should do when “Natural Language” speech input is recognized? There’s no magic here, you have to code your app to search for elements in the recognized speech that have relevance to that apps problem domain. The MSDN Voice Search sample app (link in the ‘Further Reading’ section at the end) is coded to recognise variants on ‘Take me to the Windows Dev Center’ and if that isn’t found falls back to a default of a search on Bing for whatever speech input was recognized. We’ll look at how this is done in just a moment.

Handing Speech Commands in your app

I’m not going to go into all the coding techniques used to handle voice activation here. Watch the BUILD video (link below in ‘Further Reading’) for an excellent overview of handling voice activation in both Silverlight and Windows Runtime apps, and download the sample app (link also below). But I will just call out how to handle Natural Language input.

The sample app contains the following code for handling this:

        /// <summary>
        /// Given a query string from the user, attempts rudimentary "natural language" string processing in an
        /// effort to derive a specific, curated action from the text; if a match is found, that curated action
        /// is taken. If not, the string is unhandled and should be handled elsewhere.
        /// </summary>
        /// <param name="query"> the query to attempt processing and action upon </param>
        /// <param name="actSilently"> whether or not to only take actions without audio feedback </param>
        /// <returns></returns>
        private bool TryHandleNlQuery(string query, bool actSilently)
        {
            // There are a variety of ways to say things like "I want to go to Windows Phone Dev Center"; let's load
            // some alternatives for the key components of this.
            string[] intentMarkers = AppResources.NaturalLanguageCommandIntentMarkers.Split(new char[] { ';' });
            string[] wpDevCenterNames = AppResources.WPDevCenterNames.Split(new char[] { ';' });

            int intentIndex = -1;
            int destinationIndex = -1;

            Uri destinationUri = null;
            string confirmationTts = null;

            // First we'll try to find a match for the "intent marker," e.g. "go to"
            foreach (string marker in intentMarkers)
            {
                intentIndex = query.IndexOf(marker, StringComparison.InvariantCultureIgnoreCase);
                if (intentIndex >= 0)
                {
                    break;
                }
            }

            if (intentIndex >= 0)
            {
                // Now we'll try to figure out a destination--if it comes after the intent marker in the string, we'll
                // store the destination and spoken feedback.
                foreach (string wpDevCenterName in wpDevCenterNames)
                {
                    destinationIndex = query.IndexOf(wpDevCenterName, StringComparison.InvariantCultureIgnoreCase);
                    if (destinationIndex > intentIndex)
                    {
                        destinationUri = new Uri(AppResources.WPDevCenterURL);
                        confirmationTts = AppResources.SpokenWPDevCenterSsml;
                        break;
                    }
                }
            }

            // If we found a destination to go to, we'll go there and--if allowed--provide the corresponding spoken
            // feedback.
            if (destinationUri != null)
            {
                if (!actSilently && (confirmationTts != null))
                {
                    StartSpeakingSsml(confirmationTts);
                }

                StartBrowserNavigation(destinationUri);
            }

            // If we found a destination, we handled the query. Otherwise, it hasn't been handled yet.
            return (destinationUri == null); ;
        }

That’s a lot of code, and essentially all it does is look for a number of different variants on ‘I want to go to the Windows Dev Center’. Key to understanding it is knowing that in the Resources file, the command intents and variations for the dev center name are defined:

NaturalLanguageCommandIntentMarkers: show me;go to;take me to;bring up;navigate to;launch
WPDevCenterNames: Phone Dev Center;Phone Developer Center

So all the code shown above actually does is to search for a match on the recognized text for one of the NaturalLanguageCommandIntentMarkers, followed by one of the options for WPDevCenterNames. If it matches then the app takes you to the Windows dev center. If you follow through on the sample code, you’ll see that if no match is found, then the default fallback action is just to invoke a Bing search for the recognized text.

Like I said – there’s no magic here. You just have to write code to analyse the recognized speech for strings that make sense for that app, and this could be quite a lot of code!

Parting thoughts…

Before I close, three further thoughts on Speech enabling your apps:

First, make sure you include Command definitions in your vcd file for all languages you want to support. The sample app has commands defined only for US English:

    <!-- The CommandSet Name is used to programmatically access the CommandSet -->
    <CommandSet xml:lang="en-us" Name="englishCommands">

which means that when you run it on a UK English or Australian English device, the voice commands don’t work! You have to define for each individual language you want, which could get a bit tedious. But be warned – it caught me out!

Secondly, make sure you detect when the user has typed into Cortana rather than used speech. If they typed in, don’t go speaking back to them – they typed quite possibly because they were in a meeting, so it would not be helpful to start speaking out sports results or whatever! The sample app and the BUILD video show you how to do this.

And thirdly, your user has to run the app at least once to run the code to register the speech commands with the system. Unfortunately, speech commands do not get activated just by installing the app. So the first time the user runs the app, that might be a good time to pop up a ‘Did you know?’ screen to tell them about the speech commands your app supports. All helps towards the users’ discoverability of these kind of features.

Further reading

· Video: Integrating Your App into the Windows Phone Speech Experience http://channel9.msdn.com/Events/Build/2014/2-530

· Sample: MSDN Voice Search for Windows Phone 8.1 http://code.msdn.microsoft.com/MSDN-Voice-Search-for-95c16d92

· Speech team blog post: Integrating Store Apps with Cortana in Windows Phone 8.1 http://blogs.bing.com/search/2014/04/14/integrating-store-apps-with-cortana-in-windows-phone-8-1/

Voice command element and attribute reference (Windows Phone Store apps using C#/VB/C++ and XAML)
http://msdn.microsoft.com/en-us/library/dn630431.aspx

Windows Apps JumpStart Publish Hack

Don’t forget, if you can get to London on Saturday September 20th, come to our Publish Hack at Microsoft’s Cardinal Place offices near Victoria Station and get your app published to store! Our goal for this day is to make it a fun, informal day where you can overcome those last blockers and issues that are preventing you getting over those last yards to publication. There will be fun, and pizza, and beer….oh, yes – and prizes!

We will have experts from Microsoft and from the developer community there to help you, including elite Windows App engineers from our App Consult program.

Did I mention prizes? Yes, as if the fun of taking part wasn’t enough, we’re going to give you additional incentive to take part by offering prizes, including:

- Nokia Lumia 1020 phones

- Nokia Lumia 1520 phablets

- Nokia Contactless Charging units

- Arc Mice

- Arc Keyboards

- A Goodie bag for every single developer that publishes an app on the day!

The prizes will be awarded for quality, progress made and for good old-fashioned sweat and tears.

Details and registration page:

Windows Apps JumpStart Publish Hack
Saturday 20 Sep – 9AM – 9PM
http://aka.ms/winappspublishhack 

Hope you can come along!

Windows Apps JumpStart LIVE Events Series

We’re starting a new series of events for UK developers that will help you on the journey from initial interest in developing apps for Windows 8 and Windows Phone right the way through to publishing an app to the Windows Store. You can join into the series at whichever point makes sense to you, so whether you’ve never opened Visual Studio before or written a line of C#, or you’re a developer who has already written software using out tools and languages but are new to apps, we can help you get to that point where you’re publishing your first app into our store. And we’re going to repeat the series of events every quarter, so you can jump on and off the series and attend events at whatever pace makes sense to you.

We start with two one-hour webinars run back-to-back:

Webinar 1: Windows Apps JumpStart LIVE Series – Kick Starter Webinar for the non C#/XAML developer: This one hour webinar is aimed at developers who are completely new to Visual Studio, XAML, C# and .NET. This will give you what they need to get started with downloading Visual Studio, a little bit of knowledge of C# and the structure of a XAML-based app.

Webinar 2: Windows Apps JumpStart LIVE Series – JumpStart Webinar: Building Apps with C# and XAML: This one hour webinar follows on straight away after the previous one. It is aimed at developers who are new to Windows Store app development, but who have some experience of developing software using .NET, or for those who have attended the previous webinar.

Following on from the webinars, we have a one-day dev camp:

Dev camp: Windows Apps JumpStart LIVE Series: Windows Apps Dev Workshop
This one day event will be help at Microsoft’s offices near Victoria Station, London, running 9:30 – 18:00. At this workshop, you will learn how to program Windows and Windows Phone apps, including universal apps that run on both. There will be Hands-on Labs available, and you can get tips and advice from Microsoft experts and from peers in the developer community.

And following on from that, we have a one-day Publish hackathon:

Publish event: Windows Apps JumpStart LIVE Series: Publish Hackathon
This informal one-day event will also be held at Microsoft’s building in London, running 9:00 – 21:00 on a Saturday. This is a fun 12 hour event encouraging developers to bring their existing projects (apps/cloud projects) to polish and publish through to the last mile with help from Microsoft and community experts.

First events are coming up fast!:

Come along and join us – and if not this quarter, remember we are repeating these events throughout the year so you can join us when you can!

Windows Apps JumpStart LIVE Events Series

We’re starting a new series of events for UK developers that will help you on the journey from initial interest in developing apps for Windows 8 and Windows Phone right the way through to publishing an app to the Windows Store. You can join into the series at whichever point makes sense to you, so whether you’ve never opened Visual Studio before or written a line of C#, or you’re a developer who has already written software using out tools and languages but are new to apps, we can help you get to that point where you’re publishing your first app into our store. And we’re going to repeat the series of events every quarter, so you can jump on and off the series and attend events at whatever pace makes sense to you.

We start with two one-hour webinars run back-to-back:

Webinar 1: Windows Apps JumpStart LIVE Series – Kick Starter Webinar for the non C#/XAML developer: This one hour webinar is aimed at developers who are completely new to Visual Studio, XAML, C# and .NET. This will give you what they need to get started with downloading Visual Studio, a little bit of knowledge of C# and the structure of a XAML-based app.

Webinar 2: Windows Apps JumpStart LIVE Series – JumpStart Webinar: Building Apps with C# and XAML: This one hour webinar follows on straight away after the previous one. It is aimed at developers who are new to Windows Store app development, but who have some experience of developing software using .NET, or for those who have attended the previous webinar.

Following on from the webinars, we have a one-day dev camp:

Dev camp: Windows Apps JumpStart LIVE Series: Windows Apps Dev Workshop
This one day event will be help at Microsoft’s offices near Victoria Station, London, running 9:30 – 18:00. At this workshop, you will learn how to program Windows and Windows Phone apps, including universal apps that run on both. There will be Hands-on Labs available, and you can get tips and advice from Microsoft experts and from peers in the developer community.

And following on from that, we have a one-day Publish hackathon:

Publish event: Windows Apps JumpStart LIVE Series: Publish Hackathon
This informal one-day event will also be held at Microsoft’s building in London, running 9:00 – 21:00 on a Saturday. This is a fun 12 hour event encouraging developers to bring their existing projects (apps/cloud projects) to polish and publish through to the last mile with help from Microsoft and community experts.

First events are coming up fast!:

Come along and join us – and if not this quarter, remember we are repeating these events throughout the year so you can join us when you can!

Windows Phone 8.1 Jump Start is coming!

I’m pleased to announce that the registration page is now up for this year’s Windows Phone Jump Start!

Building Apps for Windows Phone 8.1 Jump Start will be held on 29th, 30th April and extend onto the morning of May 1st. As it says on the website: “If you’re an app developer who wants to design and build Windows Phone 8.1 apps using XAML and C#, don’t miss this exciting event. The two-and-a-half day, demo-rich course, taught by experts who have years of experience developing (and writing about the process), focuses on how to create apps for Windows Phone 8.1 in Visual Studio and how to create universal app projects that share a high percentage of code and that target both Windows and Windows Phone.”

This time around, the Jump Start is structured into 4 main parts:

Section 1: Introduction

· Session 1: Introduction to Windows Phone 8.1 (50 minutes)
Overview of the Windows Phone 8.1 developer platform, including guidance on the choices of app framework now available.

Section 2: Building Windows Runtime Apps using XAML and C#

Sessions that show how to build phone apps using WinRT XAML

· Session 2: Getting Started Building Windows Runtime Apps (50 minutes)
Fundamentals of building a WinRT XAML app for a Phone target, introducing the controls, layout, styles and theme resources, AppBar, StatusBar.

· Session 3: Page Navigation and Data Binding (25 minutes)
Navigating between pages in a Windows Runtime app, page caching and data binding

· Session 4: Lists and List Items (50 minutes)
Programming Lists, formatting list items and handling long or complex lists effectively

· Session 5: Windows Runtime App Page Layout Controls (25 minutes)
Pivot, Hub and Single Page layouts.

· Session 6: Adapting UI for Different Screens (25 minutes)
Explain the new layout system, and how to ensure your UI adapts to different phone screen sizes and orientations.

· Session 7: Windows Runtime App Lifecycle (25 minutes)
Explain the ways apps can be started, terminated and resumed.

· Session 8: Localization and Globalization in Windows Runtime Apps (25 minutes)
Making your app world-ready

Section 3: Programming Windows Runtime Platform Features (Windows Runtime XAML and Silverlight 8.1)

Programming platform features in Windows Phone 8.1 apps from either Windows Runtime Apps or Windows Phone Silverlight Apps 

· Session 9: Data Storage, Backup and Roaming (50 minutes)
All about storing data, backing app data up to the cloud and roaming data across devices

· Session 10: Contracts and Sharing Files and Data (50 minutes)
Share contract, FileOpenPicker/FileSavePicker, File & Uri associations

[Day 2]

· Session 11: Background Tasks (25 minutes)
How to run code in the background

· Session 12: Maps, Geolocation and Geofencing (25 minutes)
Maps, Location and GeoFencing

· Session 13: Networking, Mobile Services and Authentication (50 minutes)
Networking fundamentals. Includes Background Transfer Service and Web Authentication Broker 

· Session 14: Tiles, badges and toasts and Notification Center (50 minutes)
Tiles and toasts and Notification Center

· Session 15: Sensors and Proximity: NFC and Bluetooth (25 minutes)
Sensors, NFC and Bluetooth

· Session 16: Contacts and Calendar (25 minutes)
WinRT APIs for Contacts and Calendar, plus new capabilities (available on phone only) for Wallet-aware apps

· Session 17: Camera, Media, Audio and Speech (50 minutes)
Working with the camera, media and video editing

· Session 18: Enterprise LOB Apps (50 minutes)
All the new features aimed at Enterprise LOB and MDDM

· Session 19: SQLite Database (25 minutes)
How to program SQLite

· Session 20: VS Tooling and Memory Profiling (25 minutes)
Introduction to the many tools built into Visual Studio to help you develop Windows apps

[Day 3]

· Session 21: App Packaging and Publication (50 minutes)
How to package your app and get your app published in the Store

· Session 22: Best practices: Building Universal Apps for Windows Phone and Windows (50 minutes)
Guidance for building for both

Section 4: Upgrading Windows Phone Silverlight 8.0 apps to Silverlight 8.1

Programming new platform features from a Silverlight app

· Session 23: Upgrading Windows Phone Silverlight 8.0 Apps to Silverlight 8.1 (50 minutes)
Recap on why you would use Silverlight, and explanation of what issues must be considered when upgrading to Silverlight 8.1. Topics include behavioural changes introduced by the quirks API, Lifecycle changes as FAR is now the only mode of reactivation, implementing share contract and using the FileOpenPicker/FileSavePicker, Web Authentication Broker, moving from MPNS to WNS, background agents alongside background tasks.

Hope you can join us for the live event.  The event is happening in Redmond and will be from 9:00am – 5:00pm Pacific Daylight Time. We realize that doesn’t make it the most convenient timeslots for everyone, but if you can’t watch it all, we hope you will join us when you can, and then catch up with the full list of sessions when the videos are posted on Channel9 shortly after!

Registration page can be found here!

SQLiteWinRT: Opening databases shipped as app content or from an SD Card

Recently, I was asked about opening a database file inserted into a device on an SD card. I had to make a slight change to the SQLWinRT wrapper to enable it, so this post explains how, and also looks at opening databases shipped in the app package as content.

Opening databases shipped as content

You can include a prepopulated SQLite database in your Windows 8.x or Windows Phone app package. When the user installs the app, the database file ends up in the App Install folder where you can open it from your code – but only in a read-only fashion:

public static async void LoadDatabase()
{
    // Get a reference to the SQLite database
    db = new SQLiteWinRT.Database(
        Windows.ApplicationModel.Package.Current.InstalledLocation, 
        "customers.sqlite");

    await db.OpenAsync(SQLiteWinRT.SqliteOpenMode.OpenRead);

    // PRAGMA temp_files=2 causes temporary files to be created in memory
    // rather than in a physicalfile in the same folder as the database
    // Note that temp files only really get used if you are going to do 
    // complicated JOINs etc
    await db.ExecuteStatementAsync("PRAGMA temp_files=2");
}

If you want to have read-write access to the database, you must copy it from the install folder to the local folder first.

async Task CopyDatabaseAsync()
{
    // Has the database been copied already?
    try
    {
        await ApplicationData.Current.LocalFolder.GetFileAsync("customers.sqlite");
        // No exception? It exists...
        return;
    }
    catch (System.IO.FileNotFoundException)
    {   // Expected response - no-op 
    }

    StorageFile dbfile = await StorageFile.GetFileFromApplicationUriAsync(
        new Uri("ms-appx:///customers.sqlite"));
    await dbfile.CopyAsync(ApplicationData.Current.LocalFolder);
}

Opening a Database on an SD Card

A database file that you open from an SD Card is related to the previous case in that it is read-only. You cannot write to any file on an SD Card from any app in Windows Phone 8.

I haven’t researched it thoroughly, but you should be able to open a database file on removable storage in a Windows 8.x Store app. You will have to request the Removable Storage capability in the app manifest, and declare the file extensions you want to access in the app manifest as well. Then you should be able to get a StorageFile reference to the database file using techniques similar to those discussed here: http://lunarfrog.com/blog/2012/10/27/external-storage-devices/   . When I’ve tried it, I’ll update this post!

What I have tried though, is doing this on a Windows Phone 8 device that has an SD card slot, such as the Nokia Lumia 820. As with Windows 8, you have to declare the file extensions you want to access in the manifest. For example, add the following after the closing </Tokens> tag:

    <Extensions>
      <FileTypeAssociation Name="sqlite" TaskID="_default" NavUriFragment="fileToken=%s">
        <SupportedFileTypes>
          <FileType ContentType="application/sqlite">.sqlite</FileType>
        </SupportedFileTypes>
      </FileTypeAssociation>
    </Extensions>

Then in your code, you can open a read-only connection to the database as follows:

public static async void LoadDatabase()
{
    // Get a reference to the SQLite database
    ExternalStorageDevice _sdCard = 
        (await ExternalStorage.GetExternalStorageDevicesAsync())
        .FirstOrDefault();

    if (_sdCard == null)
    {
        MessageBox.Show("No SD Cards found");
        return;
    }
    else
    {
        ExternalStorageFile dbStoragefile = 
            await _sdCard.GetFileAsync("customers.sqlite");
        db = new SQLiteWinRT.Database(dbStoragefile.Path);
    }

    await db.OpenAsync(SQLiteWinRT.SqliteOpenMode.OpenRead);

    // PRAGMA temp_files=2 causes temporary files to be created in memory
    // rather than in a physicalfile in the same folder as the database 
    // Note that temp files only really get used if you are going to do 
    // complicated JOINs etc
    await db.ExecuteStatementAsync("PRAGMA temp_files=2");
}

Important: I had to modify the wrapper code posted up on http://sqlwinrt.codeplex.com in order to enable this usage. I added a new override of the Database object constructor that just takes the path to the database file as a string. Earlier versions had one override of the constructor that took a Windows.Storage.StorageFile parameter and another that took a Windows.Storage.StorageFolder and the filename as string, but neither of those were usable since the ExternalStorageFile object is not as you might expect a derivative of Windows.Storage.StorageFile, but is instead in the Microsoft.Phone.Storage namespace – hence I had to create a new constructor.

Download the latest version of the wrapper code from http://sqlwinrt.codeplex.com to get this update – note that this is built against SQLite version 3.8.2 – make sure you update to this version using the Visual Studio – Tools – Extensions and Updates wizard.

Download the sample project here

Note that it covers both scenarios covered in this post. At the top of app.xaml.cs there is a compile time symbol #define USE_SDCARD – comment this out to test the ‘database shipped as content’, or leave it in to test the SD card access. You’ll need a real device for the latter and before testing the app use your PC to copy the customers.sqlite file from the project onto the sd card and insert into your phone. Sadly, the emulator does not emulate a removable storage card!

NDC London 2013

I just spent 3 enjoyable days at the NDC conference in London. NDC usually stands for Norwegian Developer Conference which happens every June in Oslo, Norway and which is a popular conference – I really hope that I can speak there next June! The organisers of that, NDC Conferences, decided to bring their show to London – hence this weeks’ show – with NDC modified slightly to stand for ‘New Developer’s Conference’

And what a show! – great speakers, great food, large, comfortable venue, loads of sessions and an awesome party. Congratulations to NDC for putting on a great event, and I look forward to supporting them in future years.

My own contribution was two sessions: Windows Phone Networking Toolkit which was an update of a session I gave at TechEd this year, and which works through various topics related to effective network programming on Windows Phone, and Creating Killer Windows Phone Apps, a brand new session looking at ways to improve the quality of your apps. That went well and I look forward to repeating it somewhere soon. It looks at correct use of the animations in the Windows Phone Toolkit, programming Push Notifications using Windows Azure Notifications Hubs, localizing your app with the Multilingual Toolkit and reducing your memory footprint to improve your apps performance on low memory devices.

The slides for the sessions are available here: