VB Magic


Multiple Database Context Mess and Intercepting writes on a WCF Data Service

Filed under: .NET, Azure, VB.NET — Tags: , , , , , , , — vbmagic @ 3:00 pm


Another weekend on hacking code to get the Tyrannt project off the ground. Again I am concentrating on the middle OData web service tier. One of the rules I set myself was to not allow this tier to update the database. These update requests are only meant to be done on the back end tier via messages passed along the Azure Service Bus (ASB).

After an initial hiccup which resulted in my WCF service request falling over with “Not enough memory” (These are hosted in Azure extra small compute instances). I managed to get a working service in the cloud that exposed all my current tables.

Next I wanted to split the tables over multiple data services. I initially achieved this by creating multiple database contexts. This also allowed me to intercept the SaveChanges call in the database context when someone did a Post or Put (Below is my current thinking of how to do this, although this may change when I find it doesn’t work 😉 )

    Public Overrides Function SaveChanges() As Integer
        For Each change In ChangeTracker.Entries
            Dim job As tJob
            Dim entity = change.Entity
            Dim entityType = change.Entity.GetType.Name
            Select Case entityType
                Case "NewsArticle"
                    job = New tjUpdateNewsArticle
                    Dim na As NewsArticle = CType(entity, NewsArticle)
                    '.... Etc
            End Select

        Return MyBase.SaveChanges()
    End Function

But when it came to trying to actually run this, I kept getting an error saying:

The model backing the 'TyranntSubsetContext' context has changed since the database was created.

After a lot of searching it seemed like some people said this was possible and other said it was not. I decided to change tack and use a single database context that had all my tables in it, but make a duplicate one in my service project which I can use to intercept the saves. (The Back end tier will need normal database access as this will be doing the writes)

Anyway I still needed to expose different tables in different services. And as a nice surprise this time, it was very easy to do.

Here is my DB Context class:

Imports System.Data.Entity
Imports Tyrannt.Model.Email
Imports Tyrannt.Model.News
Imports Tyrannt.Model.Errors
Imports Tyrannt.Model.Membership
Imports Tyrannt.Infrastructure.Jobs
Imports Tyrannt.Infrastructure.Jobs.News

Public Class GlobalDbContext
    Inherits DbContext

    Public Property NewsArticles As DbSet(Of NewsArticle)
    Public Property EmailMessages As DbSet(Of EmailMessage)
    Public Property ErrorMessages As DbSet(Of ErrorMessage)
    Public Property Members As DbSet(Of Member)
    Public Property MemberTypes As DbSet(Of MemberType)

    'Public Overrides Function SaveChanges() As Integer
    '    For Each change In ChangeTracker.Entries
    '        Dim job As tJob
    '        Dim entity = change.Entity
    '        Dim entityType = change.Entity.GetType.Name
    '        Select Case entityType
    '            Case "NewsArticle"
    '                job = New tjUpdateNewsArticle
    '                Dim na As NewsArticle = CType(entity, NewsArticle)

    '        End Select
    '    Next

    '    Return MyBase.SaveChanges()
    'End Function

End Class

And this is my service code (For now I only want to expose the NewsArticles one) :

Imports System.Data.Services
Imports System.Data.Services.Common
Imports System.Linq
Imports System.ServiceModel.Web

Public Class News
    Inherits DataService(Of GlobalDbContext)

    ' This method is called only once to initialize service-wide policies.
    Public Shared Sub InitializeService(ByVal config As DataServiceConfiguration)

        ' Expose only the required tables with the relevant access rights
        config.SetEntitySetAccessRule("NewsArticles", EntitySetRights.All)

        ' General settings
        config.UseVerboseErrors = True
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V3
    End Sub

End Class

And it worked 🙂

For those curious, here is the service URI:


I cannot guarantee this service will always work or still exist in future but it’ll be there while I test client side code.


Decisions Decisions – Solution Structure and how to get things done

Filed under: .NET, Azure, Learning — Tags: , , , , , , , — vbmagic @ 2:44 pm

I spent most of this weekend, researching different ways to achieve my project. There are certain things that I know I want to do:

Adopt a Three Tier Approach.

  1. Multiple Clients (Including a Website)
  2. A middle service layer that all the clients talk to which can read from the database but not update.
  3. A back end which is the only part of the solution allowed to update the database

The communication between the middle tier and the back end tier will be via the Azure Service bus.

I want the user of the application to be able to log in to the various client using their Windows Account/Google/Facebook ID’s.

With this in mind I was looking into ways to achieve these goals. I was mainly concentrating on what the middle service tier should be.

I wanted something that will make it easy to implement clients on non Microsoft platforms which drew me towards using the new ASP.Net MVC Web API. The downside of this is there is no metadata to describe the service which makes the client much harder to write. After a few discussions with a colleague, they suggested using WCF OData service which should allow making clients easy on the windows side. (Automatic generation of model classes when adding a service reference)

But I’m still open to suggestions from anyone reading this if they think there may be a better way.

The solution structure for the project as it stands now will be:

Tyrannt.Model (Holds the Code First EF Classes)

Tyrannt.Backoffice (Windows Azure Worker Role – The back end tier)

Tyrannt.OData (Web Role – the Middle WCF OData tier)

Tyrannt.Website (First client – will consume the OData service)

I would also like to make use of database migrations that are in Entity Framework 5. Which also brings up another question which it’s been hard to answer. Where is the best place to put the Database context. An easy solution would be to put one in both the Backoffice and OData projects but that means maintaining two separate classes that basically do the same job)

I’ve seen some posts about not having the database context in the same project as the Model classes. I’m trying to work out the best way to do this and still allow database migrations.

Feel free to comment this posts with suggestions ideas or even telling me that I’ve got it all wrong ;-).


Azure Queues

I’ve been learning about using Queues in Azure and thought the best way to learn was to start writing an application to test this. My example site will probably only run on single instances; but I decided from the start to write something that will be scalable so came up with the following principle

The client/website can only READ from the database. Anything that requires updating the database must be done via the back end. The client/website must be able to cope with the fact that the requested update will not happen immediately.

The upshot of this, I came up with a job system that uses Azure queues to pass work to the back end worker role. For the moment, I decided to have one queue which handled multiple types of jobs. This lead to me creating a job base class which will cover the common things that the different jobs required. I also decided to use an XML format to describe the job. I need to make sure though that the jobs do not grow bigger than the size allowed for an Azure Queue job (8k).

Below is the base class which is inherited in the actual job classes:

Imports Microsoft.WindowsAzure
Imports Microsoft.WindowsAzure.StorageClient
Imports Microsoft.WindowsAzure.ServiceRuntime

Public MustInherit Class tjob

    ' Enable database access
    Protected Friend _db As New TyranntDB
    ' A place holder for the user information
    Protected Friend _usr As tUser
    ' setup variables to allow for access to Azure Queues
    Protected Friend _storageAccount As CloudStorageAccount
    Protected Friend _jobQueue As CloudQueue

    ' A sample of a job entry in the Queue
    Private sample = <job type="email" user="102">

    ' the user ID which is used to pull the user information
    Private _userID As Int64

    ' Initialise the job from the XML String
    Public Sub New(jobStr As String)
        Dim jobXML As XElement = XElement.Parse(jobStr)
        _JobType = jobXML.@type
        Dim usrstr As String = jobXML.@userid
        UserID = Convert.ToInt64(usrstr)
    End Sub

    ' Create a blank job, this is used for creating a job to
    ' put onto the queue.
    Public Sub New()
        _JobType = ""
        _userID = -1
    End Sub

    ' Job type. Used to create the correct object.
    Public Property JobType As String

    ' The user ID. If this is being set then it
    ' will look up the user from the database
    Public Property UserID As Integer
            Return _userID
        End Get
        Set(value As Integer)
            _userID = value
            If _userID > 0 Then
            End If
        End Set
    End Property

    ' This is the code that "Processes" the job. Each job type must
    ' implement this code.
    Public MustOverride Function Process() As Boolean

    ' A general variable for storing any errors that
    ' occur. If it's empty then no errors are assumed.
    Public Property ErrorMessage As String

    ' This will generate an XML element that describes the job.
    Public MustOverride Function ToXML() As XElement

    ' This will generate a string version of the XML
    ' which describes this job.
    Public Overrides Function ToString() As String
        Return ToXML.ToString
    End Function

    ' This routine will pull the user information from the
    ' database and store the user detals in the _usr object.
    Protected Friend Sub GetUserDetails()
        Dim q = From u In _db.users
              Where u.ID = _userID
              Select u
        If q.Count > 0 Then
            _usr = q.Single
        End If
    End Sub

    ' If the job is being created. This function will add the job
    ' to the Azure Queue.
    Public Sub AddJobToQueue()
        ' Get the azure storage account object.
        _storageAccount = CloudStorageAccount.Parse( RoleEnvironment.GetConfigurationSettingValue(TyranntSupport.Constants.STORAGE_CONNECTION))
        ' Now get the queue client.
        Dim client As CloudQueueClient = _storageAccount.CreateCloudQueueClient
        _jobQueue = client.GetQueueReference(Constants.QUEUE_JOBS)
        ' Create the queue if it doesn't exist.
            ' Now add the job details to the queue.
            Dim msg As New CloudQueueMessage(Me.ToString)
        Catch ex As Exception
            _ErrorMessage = ex.Message
        End Try
    End Sub
End Class

The TyranntDB class is an Entity Framework code first class that allows access to the SQL Azure database. Any class prefixed with a t (E.g. tUser) is a database table class. You will notice a Sample variable using XML Literals. This is just as an example to show how that job is formed in XML. Each inherited class will have it’s own sample.

All job classes need to be able to add themselves to the Azure Queue. They also need to be able to access the SQL Azure database. These features were written into the base class. All job classes must also have a way of being “Processed” which meant adding a MustOverride function called Process. They must also be able to export themselves as an XML document which is why the MustOverride function called ToXML is added.

The website uses Forms Authentication to enable it to validate users. But I also have a site specific user table to add extra information too. As this involves updating the database, this is the first “Job” that needs creating:

Public Class tjNewUser
    Inherits tjob

    ' an example of a new user job
    Private sample = <job type="newuser" user="-1">
                         <user name="{username}" email="{user)@{domain}">Full Name</user>

    ' Extra data required by this class
    Public Property userName As String
    Public Property email As String
    Public Property fullName As String

    Public Sub New(jobStr As String)
        ' initialise basic information
        Dim jobXML As XElement = XElement.Parse(jobStr)
        ' now initialise new user information
        _userName = jobXML...<user>.@name
        _email = jobXML...<user>.@email
        _fullName = jobXML...<user>.Value
    End Sub

    Public Sub New()
        ' initialise the base information
        JobType = "newuser"
        _userName = ""
        _email = ""
        _fullName = ""
    End Sub

    ' Create the new user in the database
    Public Overrides Function Process() As Boolean
        ' first check to see if the user already exists

        Dim r = From u In _db.users
              Where u.username = _userName
              Select u

        If r.Count > 0 Then
            ' User already exists so do not continue
            ' return true in this case as request
            ' has been processed more than one.
            Return True
        End If
        ' create a new user
        Dim usr As New tUser
        ' populate the generic information
        usr.username = _userName
        usr.email = _email
        usr.fullname = _fullName
        ' now set the user group to be member
            Dim grp As tUserGroup = _db.GetUserGroup("member")

            If IsNothing(grp) Then
                usr.usergroup = _db.GetUserGroup("member")
                usr.usergroup = grp
            End If
        Catch ex As Exception
            ErrorMessage = ex.Message
            Return False
        End Try

        ' now save the user
        Catch ex As Exception
            ErrorMessage = ex.Message
            Return False
        End Try

        ' Now that the user was sucessfully created,
        ' generate a new user email job
        Dim jb As New tjEmail
        jb.EmailType = "newuser"
        jb.From = "mail@me.uk"
        jb.UserID = usr.ID
        ' Add the job to the Azure job queue
        If jb.ErrorMessage = "" Then
            Return True
            ErrorMessage = jb.ErrorMessage
            Return False
        End If
    End Function

    Public Overrides Function ToXML() As XElement
        Return <job type="newuser" userid=<%= UserID %>>
                   <user name=<%= _userName %> email=<%= _email %>><%= _fullName %></user>
    End Function

End Class

I’ve added the extra properties required for adding a new user and the extraction of these properties from the XML. The process function is also created which will create the user row in the users table. Hopefully the comments in the code should explain the process to do this. This routine also makes use of XML Literals which is a VB only thing at time of writing. (For example used in the ToXML and New functions.

As you can see at the end of the processing, we need to send a confirmation email to the user who has created the account. This kind of thing is also ideal for the back end to deal with hence being handled by the job queue system:

Imports System.Net.Mail

Public Class tjEmail
    Inherits tjob

    ' a sample email job
    Private sample = <job type="email" user="102">
                         <email from="mail@me.uk" type="newuser"/>

    ' setup extra information required by this job
    Private _from As String
    Private _emailType As String

    ' The is the from email address
    Public WriteOnly Property From As String
        Set(value As String)
            _from = value
        End Set
    End Property

    ' This will be the email type e.g. newuser
    Public WriteOnly Property EmailType As String
        Set(value As String)
            _emailType = value
        End Set
    End Property

    ' If the job XML already exists this will set up
    ' the information automatically
    Public Sub New(jobStr As String)
        Dim jobXML As XElement = XElement.Parse(jobStr)
        _from = jobXML...<email>.@from
        _emailType = jobXML...<email>.@type
    End Sub

    ' Create an empty email job if creating a new job
    Public Sub New()
        JobType = "email"
        _from = ""
        _emailType = ""
    End Sub

    ' Send the email
    Public Overrides Function Process() As Boolean
        Dim email As MailMessage
        ' Generate the correct body of the email
        Select Case _emailType
            Case "newuser"
                email = GenerateNewUserEmail()
            Case Else
                ErrorMessage = String.Format("Email Type [{0}] not recognised", _emailType)
                Return False
        End Select

        ' Now set up the SMTP server client to send the email.
        Dim smtp As New SmtpClient(My.Resources.smtpServer, Integer.Parse(My.Resources.smtpPort))
        ' Pull the smtp login details.
        smtp.Credentials = New Net.NetworkCredential(My.Resources.smtpUser, My.Resources.smtpPass)
        Catch ex As Exception
            ErrorMessage = ex.Message
            Return False
        End Try
        Return True
    End Function

    ' This will generate the subject and body of the newuser email
    Private Function GenerateNewUserEmail() As MailMessage
        Dim email As New MailMessage(_from, _usr.email)
        email.Subject = My.Resources.Resources.TyranntAccountCreated
        email.BodyEncoding = System.Text.Encoding.Unicode
        email.IsBodyHtml = False
        email.Body = String.Format(My.Resources.newUserEmail, _usr.username)
        Return email
    End Function

    Public Overrides Function ToXML() As XElement
        Return <job type="email" userid=<%= UserID %>>
                   <email from=<%= _from %> type=<%= _emailType %>/>
    End Function

End Class

The process function in this job will generate an email and pull the smtp server details out of a resource file and depending on the type of email, will take the email body from resources too.

Now these job classes are created, they can be added to the job queue by using {job}.AddJobToQueue() method as shown in the tjNewUser class. But how will they be processed. This is where the WorkerRole comes into play. As all the work is done by the job classes themselves, only a very simple piece of code is required to process the queues:

    Public Overrides Sub Run()

        Trace.WriteLine("TyranntDogsbody entry point called.", "Information")

        ' Loop forever
        While (True)
            ' Get the next message from the queue
            Dim msg As CloudQueueMessage = Nothing
            msg = _jobQueue.GetMessage(TimeSpan.FromSeconds(30))

            If IsNothing(msg) Then
                ' If message doesn't exist then seep for 1 minute
                ' Message exists so process the message
            End If
        End While
    End Sub

    Private Sub ProcessMessage(msg As CloudQueueMessage)

            ' Turn the message into an XML element
            Dim xmlMsg As XElement = XElement.Parse(msg.AsString)
            ' Extract the message type from the element
            Dim type As String = xmlMsg.@type

            ' Now we create a job
            Dim job As tjob
            Select Case type
                ' Use the message type to see what kind of job is required
                Case "newuser"
                    job = New tjNewUser(msg.AsString)
                Case "email"
                    job = New tjEmail(msg.AsString)
                Case Else
                    Exit Sub
            End Select
            ' Process the job.
            If job.Process() = True Then
                ' The job succeeded so write a trace message to say this and
                ' delete the message from the queue.
                Trace.WriteLine(String.Format("{0} succeeded", type), "Information")
                ' The job failed so write a trace error message saying why the job failed.
                ' This will leave the job on the queue to be processed again.
                Trace.WriteLine(String.Format("{0} failed: {1} ", type, job.ErrorMessage), "Error")
            End If
        Catch ex As Exception
            ' something big has gone wrong so write this out as an error trace message.
            Trace.WriteLine(String.Format("Failed to parse xml message: [{0}]", msg.AsString), "Error")
            Exit Sub
        End Try
    End Sub

As you can see from the above code. There very little extra code required to process the job as all the work is done inside the job class.

This may be refined in the future but I hope it’s helpful for some people.



Getting an ASP.NET MVC Razor site into the cloud

Filed under: ASP.NET MVC, Azure, Learning — Tags: , , , , , — vbmagic @ 12:53 pm

I started to work on a web role to test out experiments with ADO.net code first and ASP.NET MVC 3 with Razor. Out of the box, VS 2010 does not support ASP.net MVC 3 so I created the Azure project, then added a ASP.NET MVC 3 project separately. I then added that MVC project to the Azure project as a web role. Clicked run and all seemed to work fine.

But as with all things Azure, if it works in the developer environment, don’t expect it to automatically work in the cloud.

It didn’t. But I was expecting that. It was a bit hard to track down what the problems were as it only manifested as a web role that never seemed to get started.

Once I had worked out that meant it was missing core components for the web site, I was pleasantly surprised was how easy it was to fix this.

On your MVC 3 project right click the project solution and then select “Add Deployable Dependencies”

Then click the “ASP.NET MVC” and the “ASP.NET Web Pages with Razor syntax” check boxes and click ok.

You will then find a directory added to your project called _bin_deployableAssemblies.

All I did after this was “Deploy” the Azure Project and after waiting for the project to deploy into the cloud, it all started ok and worked.

I wish all problems were as easy to solve as this 🙂



Trying to get Entity Framework Code First working : The Solution

Filed under: ASP.NET MVC, Azure, Learning — Tags: , , — vbmagic @ 11:39 am


In the light of day the problem is solved. It seems that if the database you are connecting to already exists, then it will not create any new tables that you may try to access.

I basically dropped my local database and re-ran the webserver and it created the tables and the database with no issues.

On with the MVC3 course…



Trying to get Entity Framework Code First working : The Problem

Filed under: ASP.NET MVC, Azure, Learning — Tags: , , , , — vbmagic @ 10:44 pm

I’ve added reference to EntityFramework (Downloaded by NuGet) to my MVC 3 project.

I then created the following two classes to represent my tables:

Imports System.ComponentModel.DataAnnotations

Public Class tImage
    Public Property ID As Int64

    Public Property name As String

    Public Property URL As String

    Public Property type As tImageType

    Public Property tag As String
End Class
Imports System.ComponentModel.DataAnnotations

Public Class tImageType
    Public Property ID As Integer

    Public Property name As String

    Public Property description As String

End Class

And then created the following class as my data context:

Imports System.Data.Entity

Public Class TyranntDB
    Inherits DbContext

    Public Property images As DbSet(Of tImage)
    Public Property imageTypes As DbSet(Of tImageType)

End Class

I then added the following connection string to my web.config file:

    <add name="TyranntDB"
         connectionString="Data Source=.\SQL08;Initial Catalog=jsundb;Integrated Security=True;Pooling=False"
         providerName="System.Data.SqlClient" />

Everything from this point compiled ok. After this I created an Image Controller

Namespace TyranntServices
    Public Class ImageController
        Inherits System.Web.Mvc.Controller

        Dim _db As New TyranntDB
        ' GET: /Image

        Function Index() As ActionResult

            Dim model = _db.images

            Return View(model)
        End Function

And the following View to list Images (Well it was mostly auto generated by using the Add View option with list template):

@ModelType IEnumerable(Of TyranntServices.tImage)

    ViewData("Title") = "Index"
End Code


    @Html.ActionLink("Create New", "Create")

@For Each item In Model
            @Html.ActionLink("Edit", "Edit", New With {.id = item.ID}) |
            @Html.ActionLink("Details", "Details", New With {.id = item.ID}) |
            @Html.ActionLink("Delete", "Delete", New With {.id = item.ID})

This all seems to compile well. When I run though, the tables do not get generated and I get an error stating that the dbo.tImages does not exist.

It’s late now so will post this and hope I can soon update it with what is wrong.


Blog at WordPress.com.