Augmented Reality, Windows 8, and Cloud Computing
| Title | Description | Link |
|---|---|---|
| Augmented Reality applications need a Web Service Back-End. Here is a 90-Day No obligation, totally free offer to use Windows Azure as your web service for Windows 8 Clients. | You get: Compute / 750 small compute hours per month, Web sites / 10 web sites, Mobile services / 10 mobile services, Relational database / 1 SQL database, SQL reporting / 100 hours per month, Storage / 35GB with 50,000,000 storage transactions, Bandwidth / unlimited inbound & 25GB outbound, CDN / 20GB outbound with 500,000 transactions, Cache / 128MB, Service bus / 1,500 relay hours and 500,000 messages | http://www.microsoft.com/click/services /Redirect2.ashx?CR_CC=200114759 ![]() |
| Step 0: What we will build. Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code | High level introduction to our finished application. | http://blogs.msdn.com/b/brunoterkaly/archive/2012 /11/05/step-0-what-we-will-build- augmented-reality-windows-8-and-cloud-computing-how-to-implement-with-real-code.aspx# |
| Step 1–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code | Introduction. What is Augmented Reality | http://blogs.msdn.com/b/brunoterkaly/archive/2012/10/29/ step-1-augmented-reality-windows-8-and-cloud-computing- how-to-implement-with-real-code.aspx# |
| Step 2–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code | Building the first part of our Azure back-end. | http://blogs.msdn.com/b/brunoterkaly/archive/2012/10/30/ step-2-augmented-reality-windows-8-and- cloud-computing-how-to-implement-with-real-code.aspx# |
| Step 3–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code (Implementing the Cloud Back-End) | This post provides all source code and explanation for the Azure back-end. This is the back-end for the augmented reality Windows 8 Client. | http://blogs.msdn.com/b/brunoterkaly/archive/2012/11/05 /step-3-augmented-reality-windows-8-and-cloud-computing-how- to-implement-with-real-code-implementing-the-cloud-back-end.aspx# |
| Step 4–Augmented Reality, Windows 8, and Cloud Computing–How to implement with real code ... | This post provides all source code and explanation for the Windows 8 Client (Augmented Reality Demo). The augmented reality Windows 8 Client calls into the Azure back-end described in step 3 above. | http://blogs.msdn.com/b/brunoterkaly/archive/2012/11/06/ step-4-augmented-reality-windows-8-and-cloud-computing-how-to-implement-with-real-code-implementing-the- windows-8-client.aspx# |
| Source Code - Web Service Back End | This is the Windows Azure Project that Windows 8 Clients call into | http://sdrv.ms/Qoqb1J |
| Source Code - Windows 8 Client | This is the Augmented Reality Windows 8 Client | http://sdrv.ms/T38uBC |
- It is running on a Windows 8 OS and was built with Visual Studio 2012
- This post will build this application from scratch
- It will communicate with a cloud back-end that we created in the last post
-
The cloud back end is a Windows Azure Cloud Project
- It was deployed to a data center
- This post relies on the last post, which was the Front-End Web Services Tier. It was a Windows Azure Cloud Application (with a web role of type ASP.NET Web API)
-
The Windows 8 Augmented Reality Client will not run unless the cloud back-end
exists
- The cloud back-end could be running in a MS data center
- Or it could be running as a Visual Studio 2012 Project using the Windows Azure Compute Emulator
- From Visual Studio, select File/New Project/Windows Store Application
- Select Blank Application
-
You are looking a Solution Explorer
- It is available from the VIEW menu
- Right mouse click on Package.appmanifest and choose VIEW DESIGNER. This will allow us to change the capabilities. For example, we want to allow internet connectivity.
-
I selected the options you see above.
- Internet, Microphone, Pictures Library Access, Videos Library Access, Webcam
- When you run your Windows 8 application, you still need to confirm the use of your webcam and microphone
- You can also use EXPRESSION BLEND to edit your graphical interface
- Visual Studio 2012 also has a designer, available from the VIEW MENU.
- The user interface is just a bunch of XAML controls
- It is just a big XML file, similar to an Android user interface
- StackPanels and Grids make up most of the interface
- Buttons, TextBlocks, Images, Rectangles, Canvases, Radio Buttons are nested inside of StackPanels and Grids
- There is also a CapturePanel for displaying a live video feed from the web cam
-
This screen shows one of the StackPanels
- It contains nested StackPanels
-
It contains 2 buttons and the Radio Buttons for the GPS Simulator
- The GPS Simulator lets you choose San Francisco, Reno, Denver, or Death Valley
- My system does not have a built-in simulator
- The Radio buttons are stacked within a StackPanel
- The augmented reality section is simply a grid with borders
-
The grid contains TextBlocks where we will put the data that we get from calling
our Azure cloud back-end, cloud service
- Populating the TextBlocks with location and weather information is done in code-behind (MainPage.xaml.cs)
<Page x:Class="AugmentedRealityClient.MainPage" IsTabStop="false" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:local="using:AugmentedRealityClient" xmlns:d="http://schemas.microsoft.com/expression/blend/2008" xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006" mc:Ignorable="d" HorizontalAlignment="Center" VerticalAlignment="Center"> <Grid Background="{StaticResource ApplicationPageBackgroundThemeBrush}" x:Name="LayoutRoot" HorizontalAlignment="Left" VerticalAlignment="Center"> <!—Two rows, One column--> <Grid.RowDefinitions> <RowDefinition Height="165"/> <RowDefinition/> </Grid.RowDefinitions> <Grid.ColumnDefinitions> <ColumnDefinition/> </Grid.ColumnDefinitions> <Border BorderThickness="3" BorderBrush="White" Grid.Row="0"/> <StackPanel Grid.Row="0" Orientation="Horizontal" Margin="6" HorizontalAlignment="Left" VerticalAlignment="Center"> <Button x:Name="butConfigure" Content="Configure" Width="200" Height="80" FontSize="32" Margin="6" Click="ConfigureCamera_Click"/> <Button x:Name="butTurnCameraOn" Content="Take Picture" Width="218" Height="80" FontSize="32" Margin="6,6"/> <StackPanel> <TextBlock x:Name="txtGPS" FontSize="14" Canvas.Left="22" Canvas.Top="9" Margin="0,12,0,2" Foreground="Yellow" FontFamily="Segoe UI" Text="GPS Simulator" FontWeight="Bold" HorizontalAlignment="Left" VerticalAlignment="Top" TextAlignment="Center"/> <RadioButton Name="radSF" GroupName="GPS" Content="{Binding Path=City1, Mode=TwoWay}" IsChecked="{Binding Path=IsChecked1, Mode=TwoWay}" Checked="radSF_Checked"> </RadioButton> <RadioButton Name="radReno" GroupName="GPS" Content="{Binding Path=City2, Mode=TwoWay}" IsChecked="{Binding Path=IsChecked2, Mode=TwoWay}" Checked="radReno_Checked"> </RadioButton> <RadioButton Name="radDenver" GroupName="GPS" Content="{Binding Path=City3, Mode=TwoWay}" IsChecked="{Binding Path=IsChecked3, Mode=TwoWay}" Checked="radDenver_Checked"> </RadioButton> <RadioButton Name="radDeathValley" GroupName="GPS" Content="{Binding Path=City4, Mode=TwoWay}" IsChecked="{Binding Path=IsChecked4, Mode=TwoWay}" Checked="radDeathValley_Checked"> </RadioButton> </StackPanel> </StackPanel> <StackPanel Grid.Row="1" HorizontalAlignment="Center" VerticalAlignment="Center"> <Canvas x:Name="previewCanvas1" Background="Black" Width="640" Height="600"> <CaptureElement x:Name="previewElement1" HorizontalAlignment="Left" VerticalAlignment="Top"/> <Grid HorizontalAlignment="Left" VerticalAlignment="Top" Margin="5"> <Grid.ColumnDefinitions> <ColumnDefinition Width="100"/> <ColumnDefinition Width="300"/> <ColumnDefinition Width="*"/> </Grid.ColumnDefinitions> <Grid.RowDefinitions> <RowDefinition Height="20"/> <RowDefinition Height="20"/> <RowDefinition Height="20"/> <RowDefinition Height="20"/> <RowDefinition Height="20"/> </Grid.RowDefinitions> <!--Row 1--> <Rectangle Grid.Column="0" Grid.Row="0" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock Grid.Column="0" Grid.Row="0" Text="Neighborhood" HorizontalAlignment="Center" VerticalAlignment="Center" Foreground="#FFF2F213"/> <Rectangle Grid.Column="1" Grid.Row="0" Stroke="white" Fill="Black" Opacity=".3"/> <TextBlock x:Name="bus_and_neighborhood" Grid.Column="1" Grid.Row="0" HorizontalAlignment="Center" VerticalAlignment="Center"/> <!--Row 2--> <Rectangle Grid.Column="0" Grid.Row="1" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock Grid.Column="0" Grid.Row="1" Text="Elevation" HorizontalAlignment="Center" VerticalAlignment="Center" Foreground="#FFF2F213"/> <Rectangle Grid.Column="1" Grid.Row="1" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock x:Name="elevation" Grid.Column="1" Grid.Row="1" Text="(1,0)" HorizontalAlignment="Center" VerticalAlignment="Center"/> <!--Row 3--> <Rectangle Grid.Column="0" Grid.Row="2" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock Grid.Column="0" Grid.Row="2" Text="Lat/Long" HorizontalAlignment="Center" VerticalAlignment="Center" Foreground="#FFF2F213"/> <Rectangle Grid.Column="1" Grid.Row="2" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock x:Name="latlong" Grid.Column="1" Grid.Row="2" Text="(1,0)" HorizontalAlignment="Center" VerticalAlignment="Center"/> <!--Row 4--> <Rectangle Grid.Column="0" Grid.Row="3" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock Grid.Column="0" Grid.Row="3" Text="Max temp" HorizontalAlignment="Center" VerticalAlignment="Center" Foreground="#FFF2F213"/> <Rectangle Grid.Column="1" Grid.Row="3" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock x:Name="max_temp" Grid.Column="1" Grid.Row="3" Text="(1,0)" HorizontalAlignment="Center" VerticalAlignment="Center"/> <!--Row 5--> <Rectangle Grid.Column="0" Grid.Row="4" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock Grid.Column="0" Grid.Row="4" Text="MinTemp" HorizontalAlignment="Center" VerticalAlignment="Center" Foreground="#FFF2F213"/> <Rectangle Grid.Column="1" Grid.Row="4" Stroke="White" Fill="Black" Opacity=".3"/> <TextBlock x:Name="min_temp" Grid.Column="1" Grid.Row="4" Text="(1,0)" HorizontalAlignment="Center" VerticalAlignment="Center"/> <Image Grid.Column="2" Grid.Row="0" Grid.RowSpan="5" Source="http://www.ssec.wisc.edu/data/us_comp/image1.jpg" Opacity=".7"/> </Grid> </Canvas> </StackPanel> </Grid> </Page>- The ordering of controls is critical.
- It allows us to overlay data on top of the live video stream from the web cam.
- If you notice, many of the TextBlocks appear AFTER the CaptureElement control.
- This controls the z-order of controls.
| Line | Comment |
|---|---|
| 35-49 | The two buttons. One button is for turning on the webcam and the other for taking a photo. I've illustrated photo taking in a previous post. Contains a reference to an event method (ConfigureCamera_Click) |
| 50-91 | The user interface for the GPS simulator. A TextBlock and a bunch of Radio Buttons. All 4 radio buttons have an event procedure because they end up calling into the Azure Web Service for a new GPS location. |
| 98 | The canvas to display the augmented reality live video stream from the web cam |
| 103-105 | The control that displays the augmented reality live video stream |
| 143, 169, 196, 224, 250 | The TextBlocks that will display the augmented reality data coming from the Azure back-end cloud application. |
| 257 | Used to store the latest weather satellite image |
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using Windows.Foundation;
using Windows.Foundation.Collections;
using Windows.UI.Xaml;
using Windows.UI.Xaml.Controls;
using Windows.UI.Xaml.Controls.Primitives;
using Windows.UI.Xaml.Data;
using Windows.UI.Xaml.Input;
using Windows.UI.Xaml.Media;
using Windows.UI.Xaml.Navigation;
// Some extra using statements for my code
using Windows.Devices.Enumeration;
using Windows.Media.Capture;
using Windows.Media.MediaProperties;
using System.Net.Http;
using Windows.Data.Json;
using System.Collections.ObjectModel;
using System.ComponentModel;
namespace Windows8AugmentedRealityClient
{
public sealed partial class MainPage : Page
{
private Windows.Media.Capture.MediaCapture m_mediaCaptureMgr;
// Our GPS simulator
GPSSimulator _gps_simulator;
// This object get populated with data from calling into
// our Windows Azure Web Service, which was created in the
// previous blog
private LocationInfo loc_info = new LocationInfo();
public MainPage()
{
this.InitializeComponent();
// previewElement1 is our view port to see the streaming
// video content from the web cam
previewElement1.Source = null;
// Make sure it is visible
previewCanvas1.Visibility = Windows.UI.Xaml.Visibility.Visible;
// Initialize our GPS Simulator
_gps_simulator = new GPSSimulator();
_gps_simulator.City1 = "San Francisco";
_gps_simulator.Gps1 = "37.788345,-122.404679";
_gps_simulator.IsChecked1 = true;
_gps_simulator.City2 = "Reno";
_gps_simulator.Gps2 = "39.545353,-119.815631";
_gps_simulator.IsChecked2 = false;
_gps_simulator.City3 = "Denver";
_gps_simulator.Gps3 = "39.752073,-104.979968";
_gps_simulator.IsChecked3 = false;
_gps_simulator.City4 = "Death Valley";
_gps_simulator.Gps4 = "36.619937,-117.10415";
_gps_simulator.IsChecked4 = false;
// Trap some change events (for debugging purposes)
_gps_simulator.PropertyChanged += new System.ComponentModel.PropertyChangedEventHandler(gpsChanged);
// Bind our GPS Simulator to the Windows 8 form's DataContext
this.DataContext = _gps_simulator;
}
private void gpsChanged(object sender, PropertyChangedEventArgs e)
{
string s = "The property:'" + e.PropertyName + "' was changed";
}
// Ignored for this demo
protected override void OnNavigatedTo(NavigationEventArgs e)
{
}
// Hit when user hits the "Configure" button
// This method initializes the web cam to display the live
// streaming video content
internal async void ConfigureCamera_Click(Object sender, Windows.UI.Xaml.RoutedEventArgs e)
{
// Initialize Media Capture API
try
{
// Prepare Media Capture
m_mediaCaptureMgr = new Windows.Media.Capture.MediaCapture();
var _captureSettings = new Windows.Media.Capture.MediaCaptureInitializationSettings();
DeviceInformationCollection deviceInfos = await DeviceInformation.FindAllAsync(Windows.Devices.Enumeration.DeviceClass.VideoCapture);
_captureSettings = new MediaCaptureInitializationSettings();
_captureSettings.StreamingCaptureMode = StreamingCaptureMode.Video;
_captureSettings.PhotoCaptureSource = PhotoCaptureSource.VideoPreview;
if (deviceInfos.Count > 0)
{
_captureSettings.VideoDeviceId = deviceInfos[0].Id;
}
await m_mediaCaptureMgr.InitializeAsync(_captureSettings);
}
catch (Exception exception)
{
// Ignore errors for now
string s = exception.Message;
}
// Start the Preview
try
{
previewElement1.Source = m_mediaCaptureMgr;
await m_mediaCaptureMgr.StartPreviewAsync();
}
catch (Exception exception)
{
previewElement1.Source = null;
}
}
// This method performs the call into our Azure Web Service
// fromt the previous post. This method will pass GPS parameters
// to the cloud service and receive back JSON data that will
// need to be parsed.
private async System.Threading.Tasks.Task CallLocationWebService(string gps)
{
// Call into the emulator. This assumes you are running the
// cloud project from the last post in the backgruond
string _location = "http://127.0.0.1:81/api/values?location={0}";
// You can use the line below once you deploy your cloud
// application to the cloud (a MS data center)
//string _location = "http://locationwebservice.cloudapp.net/api/values?location={0}";
// Now make the aynchronous call. We need to pass the GPS
// parameters here to the _location string mentioned above.
using (HttpClient clientlocation = new HttpClient())
using (var response = await clientlocation.GetAsync(string.Format(_location, gps)))
{
if (response.IsSuccessStatusCode)
{
string webresponse = await response.Content.ReadAsStringAsync();
// Parse the string into a JSONObject
var parsedResults = JsonObject.Parse(webresponse);
IJsonValue val;
// Extract data embedded in JSONObject.
// Assign to controls in user interface
if (parsedResults.TryGetValue("latitude", out val))
loc_info.latitude = val.GetString();
if (parsedResults.TryGetValue("longitude", out val))
loc_info.longitude = val.GetString();
if (parsedResults.TryGetValue("bus_and_neighborhood", out val))
loc_info.bus_and_neighborhood = val.GetString();
if (parsedResults.TryGetValue("elevation", out val))
loc_info.elevation = val.GetString();
if (parsedResults.TryGetValue("bus_and_neighborhood", out val))
loc_info.bus_and_neighborhood = val.GetString();
if (parsedResults.TryGetValue("max_temp", out val))
loc_info.max_temp = val.GetString();
if (parsedResults.TryGetValue("min_temp", out val))
loc_info.min_temp = val.GetString();
this.bus_and_neighborhood.Text = loc_info.bus_and_neighborhood;
this.elevation.Text = loc_info.elevation;
this.latlong.Text = loc_info.latitude + "/" + loc_info.longitude;
this.max_temp.Text = loc_info.max_temp;
this.min_temp.Text = loc_info.min_temp;
}
}
}
// Checkbox events
// Called when user clicks a radio button on the GPS Simulator
private async void radSF_Checked(object sender, RoutedEventArgs e)
{
string s = _gps_simulator.Gps1;
await CallLocationWebService(s);
}
private async void radReno_Checked(object sender, RoutedEventArgs e)
{
string s = _gps_simulator.Gps2;
await CallLocationWebService(s);
}
private async void radDenver_Checked(object sender, RoutedEventArgs e)
{
string s = _gps_simulator.Gps3;
await CallLocationWebService(s);
}
private async void radDeathValley_Checked(object sender, RoutedEventArgs e)
{
string s = _gps_simulator.Gps4;
await CallLocationWebService(s);
}
// A helper method. Not currently used but was valuable during
// debugging my controls.
public IEnumerable<T> HelperFindMatch<T>(DependencyObject depObj) where T : DependencyObject
{
System.Diagnostics.Debugger.Break();
if (depObj != null)
{
for (int i = 0; i < VisualTreeHelper.GetChildrenCount(depObj); i++)
{
DependencyObject child = VisualTreeHelper.GetChild(depObj, i);
if (child != null && child is T)
{
yield return (T)child;
}
foreach (T childOfChild in HelperFindMatch<T>(child))
{
yield return childOfChild;
}
}
}
}
// A helper method. Not currently used but was valuable during
// debugging my controls.
public T HelperFindVisualChildByName<T>(DependencyObject parent, string name) where T : DependencyObject
{
for (int i = 0; i < VisualTreeHelper.GetChildrenCount(parent); i++)
{
var child = VisualTreeHelper.GetChild(parent, i);
string controlName = child.GetValue(Control.NameProperty) as string;
if (controlName.IndexOf("rad") != -1 && controlName != name)
{
return child as T;
}
else
{
T result = HelperFindVisualChildByName<T>(child, name);
if (result != null)
return result;
}
}
return null;
}
}
// This class represents the data populated by the Windows Azure
// Web Service
public class LocationInfo
{
// Data we will lookup
public string latitude { get; set; }
public string longitude { get; set; }
public string elevation { get; set; }
public string bus_and_neighborhood { get; set; }
public string max_temp { get; set; }
public string min_temp { get; set; }
}
//Implement INotifiyPropertyChanged interface to subscribe for
//property change notifications
public class GPSSimulator : INotifyPropertyChanged
{
private string _gps1;
private string _city1;
private bool _ischecked1;
private string _gps2;
private string _city2;
private bool _ischecked2;
private string _gps3;
private string _city3;
private bool _ischecked3;
private string _gps4;
private string _city4;
private bool _ischecked4;
public string Gps1
{
get { return _gps1; }
set
{
_gps1 = value;
RaisePropertyChanged("Gps1");
}
}
public string City1
{
get { return _city1; }
set
{
_city1 = value;
RaisePropertyChanged("City1");
}
}
public bool IsChecked1
{
get { return _ischecked1; }
set
{
_ischecked1 = value;
RaisePropertyChanged("IsChecked1");
}
}
public string Gps2
{
get { return _gps2; }
set
{
_gps2 = value;
RaisePropertyChanged("Gps2");
}
}
public string City2
{
get { return _city2; }
set
{
_city2 = value;
RaisePropertyChanged("City2");
}
}
public bool IsChecked2
{
get { return _ischecked2; }
set
{
_ischecked2 = value;
RaisePropertyChanged("IsChecked2");
}
}
public string Gps3
{
get { return _gps3; }
set
{
_gps3 = value;
RaisePropertyChanged("Gps3");
}
}
public string City3
{
get { return _city3; }
set
{
_city3 = value;
RaisePropertyChanged("City3");
}
}
public bool IsChecked3
{
get { return _ischecked3; }
set
{
_ischecked3 = value;
RaisePropertyChanged("IsChecked3");
}
}
public string Gps4
{
get { return _gps4; }
set
{
_gps4 = value;
RaisePropertyChanged("Gps4");
}
}
public string City4
{
get { return _city4; }
set
{
_city4 = value;
RaisePropertyChanged("City4");
}
}
public bool IsChecked4
{
get { return _ischecked4; }
set
{
_ischecked4 = value;
RaisePropertyChanged("IsChecked4");
}
}
public event PropertyChangedEventHandler PropertyChanged;
protected void RaisePropertyChanged(string name)
{
if (PropertyChanged != null)
{
PropertyChanged(this, new PropertyChangedEventArgs(name));
}
}
}
}
Thanks for reading Your assignment is to go to http://programmableweb.com and do your own augmented reality and data mash-ups.Don't forget to send it back to me so I can blog about it
(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)
Tags:






