Blog

Failure of the dynamic keyword

Sun, 18 Dec 2011 11:39:18 GMT

I would consider the dynamic keyword that was introduced in C# 4, a failure. It was really a hack to integrate dynamic languages with the CLR, a venture that Microsoft seems to have abondoned. The only two uses I know of it today is in Razor View Engine and Massive ORM.

I think that most C# developers has not even encountered dynamic at all.

The difference between the dynamic and the anonymous type, is that the anonymous type is actually compiled, and the dynamic is resolved during runtime. The real problem with anonymous types is they aren't easily returned from method calls. Consider the following.

private static object GetEmployee()
{
    return new { Name = "John", Profession = "Santa" };
}

object employee = GetEmployee();

The object employee contains the properties Name and Profession, but how do we get the data out? How do we cast the employee to the anonymous type? We have two options here

Use reflection to get the data
Use an ugly generics hack to cast the type (see line 208 of GeneratePageTypes.tt)

None of these are very convenient, and for this purpose we could use dynamic.

private static dynamic GetEmployee()
{
    dynamic employee = new { Name = "John", Profession = "Santa" };
    return employee;
}

var employee = GetEmployee();
Console.WriteLine("Employee of the month: {0}, {1}", employee.Name, employee.Profession);

This works because the compiler will always accept a call to a member on a dynamic type, but if that member does not exist during runtime, it will fail. Is it good, or bad? Your choice. This is the basics of dynamic keyword in C#.

The practical application

Have you ever been working with regular expressions? Your code probably would look like this.

string time = "12:00";
var match = Regex.Match(time, @"(?<Hour>\d{2}):(?<Minute>\d{2})");
if (match.Success)
{
    var hour = match.Groups["Hour"].Value;
    var minute = match.Groups["Minute"].Value;
    Console.WriteLine("The time is {0}.{1}", hour, minute);
}

By making the result of the regular expression dynamic we could be looking at code like this.

string time = "12:00";
var match = new DynamicRegex(@"(?<Hour>\d{2}):(?<Minute>\d{2})").Match(time);
if (match.Success)
{
    Console.WriteLine("The time is {0}.{1}", match.Hour, match.Minute);
}

It's less code, so it must be better - right? Let's take a look at the DynamicRegex class. It's just a wrapper around System.Text.RegularExpressions.Regex.

public class DynamicRegex
{
    private readonly Regex expression;
    public DynamicRegex(string expression)
    {
        this.expression = new Regex(expression);
    }

    public dynamic Match(string input)
    {
        return new DynamicMatch(expression.Match(input));
    }
}

Nothing exciting here. The match method returns a dynamic which is created by wrapping System.Text.RegularExpressions.Match in a class called DynamicMatch. And what does that look like?

public class DynamicMatch : DynamicObject
{
    private readonly Match match;

    public DynamicMatch(Match match)
    {
        this.match = match;
    }

    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        var prop = match.GetType().GetProperty(binder.Name);

        // Match instance property
        if (prop != null)
        {
            result = prop.GetValue(match, null);
            return true;
        }

        // Group value
        Group group;
        if ((group = this.match.Groups[binder.Name]) != null)
        {
            result = group.Value;
            return true;
        }

        result = null;
        return false;
    }
}

Here it becomes interesting. We can create our own dynamic object by implementing the IDynamicMetaObjectProvider interface, or inherit from System.Dynamic.DynamicObject as I've done here. There are a lot of fun methods to override, to specify what should happen when calling a method on a dynamic object. This is very much as the missing_method in Ruby.

My code is very simple. Go check if there is a property on the Match object that we're trying to call. If not, it must be a matching group we're after. This is why we can call match.Success and match.Hour, match.Minute on the same instance. So, if we have a matching group called Success, we might not get the result we're after. That is the price we pay for syntactic sugar.

What's the point?

This is just the kind of problem where we could use the dynamic keyword. Another is representing data from an IDataReader when we don't want a typed schema. We could use dynamic to get properties out of a JSON document or XML for that matter.

But these applications are things for framework developers and not mere mortals. I would say it is easier to make a mistake when you mix static types and dynamic, and you shouldn't if you don't know what you're doing. C# is a static language and as "Joe developer", you just don't have to care.

Alive v0.2 is released

Fri, 16 Dec 2011 19:52:22 GMT

During my commuting, I've been able to pick up the most critical issues with Alive and fix them. Sadly, there's no easy fix to support any other browsers than Chrome at the moment. I'm just going to say it out loud.

Alive v0.2 is Chrome only. I will attend to this issue in v0.3.

Here comes a list of changes. Note that configuration is a breaking change, and you will have to address it if you're upgrading.

The release may be downloaded from

Google Code

NuGet - LiteMedia Alive

Changes

New configuration schema

The configuration schema of version 1.1 didn't make much sense, so I sharpened it a bit. These are the major changes

No section groups, but only one section handler.
Configuration section handler name has become lower case.
Previous "group" element has been renamed to "chart".
Previous "groups" element has been renamed to "group.

Here's an example of the new schema.

  <alive>
    <settings columns="3" />
    <counters>
      <group name="Hardware">
        <chart name="CPU" updateLatency="1000">
          <counter name="CPU" categoryName="Processor" counterName="% Processor Time" instanceName="_Total" />
        </chart>
		<chart name="Memory Activity" updateLatency="1000">
		  <counter name="Memory" categoryName="Memory" counterName="Pages/sec" />
		</chart>
        <chart name="Memory Usage" updateLatency="5000">
          <counter name="RAM" categoryName="Memory" counterName="% Committed Bytes In Use" />
          <counter name="Page file" categoryName="Paging File" counterName="% Usage" instanceName="_Total" />
        </chart>
	  </group>
	  <group name="IIS">
        <chart name="ASP.NET Performance" updateLatency="1000">
          <counter name="Requests/sec" categoryName="ASP.NET Applications" counterName="Requests/Sec" instanceName="__Total__" />
        </chart>
		<chart name="Session state server" updateLatency="5000">
		  <counter name="Active" categoryName="ASP.NET" counterName="State Server Sessions Active" />
		</chart>
      </group>
    </counters>
  </alive>

Scaling 100, 250, 500, 1000

When a data point reaches above the limit, the chart will scale itself, to be able to display all the values. In previous version it would scale from 100 directly to 500 which made smaller values very hard to intepret if some data point was 101. That is why 101 now scales the chart to a max value of 250. This makes the chart easier to read.

Writing 1000 as 1K and 1000000 as 1M

Some performance counters may have very large values. In order to make it easier to read, we're now writing 1000 as 1K and 1 million as 1M.

Define max value on chart element

In the configuration you can specify the default max value for a chart. When the chart loads, it will have this as its highest value until a data point forces it to scale. This is useful when you know that you will only have values below 10.

<chart name="Shopping Cart" updateLatency="1000" max="10">
  ...
</chart>

Remote performance counters

This is a major feature. You may track counters on other computers. You do this by adding "machine" to the counter element in configuration. Here's an example.

<group name="SQL Server">
	<chart name="CPU" updateLatency="1000">
		<counter name="CPU" categoryName="Processor" counterName="% Processor Time" instanceName="_Total" machine="SQLSERVER1" />
	</chart>
</group>

That was the easy part. Now you need to make sure that your application pool identity has the required permissions to read that performance counter.

Make sure that the remote machine is running "remote registry service".
Add the application pool identity to the "Performance Monitor Users" group on the remote machine.
Use regedt32.exe to give your application pool identity read access to following keys
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\SecurePipeServers\winreg
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Perflib
Also give your application pool identity read access to the following files
%SystemRoot%\System32\Perfc009.dat
%SystemRoot%\System32\Perfh009.dat

Secure your Alive Dashboard

Wed, 14 Dec 2011 20:09:54 GMT

What kind of security is implemented in Alive?

- None

But it is essential that you don't expose Alive publicly. There are no guarantees that a hacker won't use Alive against you. So, how do you go ahead securing Alive? Easiest is to use the security model in asp.net and add the following to your web.config.

<location path="Alive.axd">
	<system.web>
		<authorization>
			<allow users="?"/>
			<deny users="*"/>
		</authorization>
	</system.web>
</location>

This will allow only authenticated users, and deny anonymous. How to turn that "Access Denied" into a proper authentication can be read on MSDN.

http://msdn.microsoft.com/en-us/library/eeyk640h.aspx

Custom Performance Counters in Alive

Sun, 11 Dec 2011 11:53:58 GMT

If you've missed previous posts about Alive, you can read more about it at

The real power of Alive, is when it comes to custom performance counters. In your application you have events that would be nice to track in real time. A new order is put, someone paid with their credit card, a user logged in. For this you can create custom performance counters.

Create a new performance counter

It is easy to create a new performance counter on your development machine. Open up server explorer in Visual Studio. Right click on Performance Counters and select "Create New Category".

This gives you a slick interface where you can create your new counter.

Create a new web application and include Alive.

PM> Install-Package LiteMediaAlive

Edit your web.config so Alive configuration looks like following.

  <Alive>
    <settings columns="3" />
    <counters>
      <groups>
        <group name="Test" updateLatency="1000">
          <counter name="Test" categoryName="Test Category" counterName="Test Increment" />
        </group>
      </groups>
    </counters>
  </Alive>

Create a new ASPX page.

<%@ Page Language="C#" AutoEventWireup="true" CodeBehind="Index.aspx.cs" Inherits="LiteMedia.Alive.Web.Test.Index" %>
<!DOCTYPE html5>
<html>
    <head>
        <title>Alive custom performance counter</title>
        <style>
            div { display: block; }
            iframe { width: 100%; height: 500px; border: none; }
        </style>
    </head>
    <body>
        <form runat="server">
        <div>
            <h1>Press the button to increment the counter</h1>
            <asp:ScriptManager runat="server">
            </asp:ScriptManager>
            <asp:UpdatePanel runat="server">
                <ContentTemplate>
                    <asp:Button runat="server" Text="Press me" OnClick="IncreaseCounter" />
                </ContentTemplate>
            </asp:UpdatePanel>
            <iframe src="/Alive.axd/" />
        </div>
        </form>
    </body>
</html>

We bind the button to a method in code behind that increases the counter.

protected void IncreaseCounter(object sender, EventArgs e)
{
    using (var counter = new PerformanceCounter("Test Category", "Test Increment", readOnly: false))
    {
        counter.Increment();
    }
}

Bam! You're done! You can download the whole example from here.

LiteMedia.Alive.CustomPerformanceCounter.zip

There are a lot of performance counter types to explore. There's an excellent article on The Code Project about performance counters.

Create performance counters in your production environment

It is a common scenario that you can't reach the production environment from your development machine. In that case I use this piece of F# to create my performance counters.

open System.Diagnostics

let args = System.Environment.GetCommandLineArgs()

printf "LiteMedia, Mikael Lundin\n"
printf "Create performance counters\n"

if args.Length <> 3 then
  printf "Usage: CreatePerfmon [category name] [category description] "

else
  let name = args.[1]
  let description = args.[2]
  
  let counterCreation = new CounterCreationDataCollection()

  let create name description counterType =
    new CounterCreationData(name, description, counterType)

  create "# operations executed" "Number of total operations executed" PerformanceCounterType.NumberOfItems32
    |> counterCreation.Add
    |> ignore

  create "# operations / sec" "Number of operations executed per second" PerformanceCounterType.RateOfCountsPerSecond32 
    |> counterCreation.Add 
    |> ignore

  create "average time per operation" "Average duration per operation execution" PerformanceCounterType.AverageTimer32
    |> counterCreation.Add
    |> ignore

  create "average time per operation base" "Average duration per operation execution base" PerformanceCounterType.AverageBase
    |> counterCreation.Add
    |> ignore
  
  PerformanceCounterCategory.Create(name, description, PerformanceCounterCategoryType.MultiInstance, counterCreation) |> ignore

Compile it into a runnable exe and execute on your production environment like this.

CreateCounter.exe "New orders" "Increments when new orders are added to the system"

This will give you a performance category on the server called "New orders" and it will contain the following counters.

# operations executed
# operations / sec
average time per operation

This is a really simple way to keep an eye on the production environment and that you're recieving new orders in the rate that you should.

Alive demo

Thu, 08 Dec 2011 06:21:16 GMT

I'm currently working on version 0.2 of Alive, but not announcing anything just yet. Expect an update in a couple of weeks. Until then I urge you to take a look at my live demo. Open it up in your favorite Chrome browser (Firefox is ok to).

You have to be a bit patient with my server, because it is a very old crappy machine in my closet, I mostly use as an development environment.

Database change management

Sun, 04 Dec 2011 17:03:23 GMT

I thought I would get started with database change management and Tarantino, but after spending a day trying to figure it out, I eventually decided that it would go much faster to just roll my own.

What is database change management?

Most developers today are familiar with source version control. Even if some use it only for backups only, the main idea is for you to go back in history when needed or to try out different approaches in other branches. Using SVC (software version control) has become a standard in the industry, but very few apply the same idea on databases even though they also are a major part of most application development.

How not to version control databases

Check in the database to source control It works as long as you are only one developer to check in the binary database file to your repository. You will however not be able to merge changes from different developers and you will have to do manual updates of your production environment when the time comes.
Share database with team members on a central server The most common way to handle database development is not to version control at all, but to setup the database on a shared central server. This fails when you need to revert to a previous version, but it also makes it harder to work as a team. When you make changes to the database you will affect your team members because they don't have your code to support the database changes. In worst case the whole team will halt production until you've checked in your code to support the database change.

Why should I care?

If you manage to version control your database development, you will not only be able to revert to old revisions, work in branches but also automatically get old databases up to date within your release process. This is much more efficient than manually merging the databases at every release, and also less error prone.

You bring stability to your release cycle
You reduce hold ups in the production line for your team
You bring greater control to your database development process

How do I get started?

If you use NAnt or MsBuild as build script you should probably head over to Tarantino and download their software since it has excellent support for your environment. If not, you may keep on reading. Since I'm using psake at the moment, I decided to implement my database change management in PowerShell v2. The concept is very simple. You keep a directory in your source control where you store database creation/update scripts. The key component is that you never ever change any of these scripts after their first commit. When you need to change something in the database you create a new script. That way you will always be able to create a new database and bring it up to head revision at any time.

The naming convention here is very important. Next thing you need is a database where you have a table and a column that tells you what version this database is. As you've already guessed, this is exactly up to what revision the change scripts has been applied.

All you need now is one function that can look at your database, decide what version it is and apply those changes that has not yet been applied. Actually, I do it in three functions and they look like this.

Update-Database $connection_string $database_directory Connects to the database and get the version number. Finds all change scripts in the database directory that are above the specific version and applies them to the database. Last it updates the database with the current version number.
Build-Database $sql_server $database_name Connects to the DBMS and creates an empty database. In this database it adds the necessary table and column to keep track of database versions.
Drop-Database $sql_server $database_name Removes the database from the DBMS.

Example usage

In my current project I want to run integration tests on a fresh database every time I check in code. That means I will have the following build process executed.

Compile
Drop-Database "MAIA\SQLEXPRESS" "IntegrationTests"
Build-Database "MAIA\SQLEXPRESS" "IntegrationTests"
Update-Database "Data Source=MAIA\SQLEXPRESS;Initial Catalog=IntegrationTests;Integrated Security=True;" ".\Database"
Run integration tests

When I check in code I will rebuild the database and execute my integration tests on it. That way I will not only test my code, but also verify that my change scripts are working. When it is time for release I will be able to run Update-Database on my production database, because I know that those build scripts has been thoroughly tested. Here is my PowerShell script if you're interested. I still see myself as a novice in PowerShell scripting, but it works and is quite minimalistic.

database-change-management.ps1

An update to this database script has been posted here.

Database versioning updated

Sun, 04 Dec 2011 17:00:47 GMT

I have been getting some feedback on my blog post about database change management, and I thought it was about time to make some updates to the database change management powershell script that I posted there. Let's go through this, step by step.

My change management procedure means that I will rebuild the database from scratch for each build. Before building the database, I need to drop the old one.

# Will drop the database if it exists
Function Drop-Database ([string]$sql_server, [string]$database_name)
{
	try {
		[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO')  | out-null
		$s = new-object ('Microsoft.SqlServer.Management.Smo.Server') $sql_server
		$s.Refresh();
		$s.KillDatabase($database_name);
		Write-Host "Killed database $databaseName"
	}
	catch {
		Write-Error "Tried to delete database $databaseName but failed, probably because it did not exist"
	}
}

We could check if the database exists before we try to drop.

# Determine if database exists
Function Exists-Database ([string]$sql_server, [string]$database_name)
{
	$exists = $FALSE
	try {
		# Connect and run a command using SMO 
		[reflection.assembly]::LoadWithPartialName("Microsoft.SqlServer.Smo")
		
		# Get server reference
		$s = new-object ("Microsoft.SqlServer.Management.Smo.Server") $sql_server
		
		foreach($db in $s.databases) 
		{
			# Database exists?
			if ($db.name -eq $database_name) {
				$exists = $TRUE
			}
		}
	}
	catch {
		Write-Error "Failed to connect to $sql_server"
	}

	# Return
	$exists
}

Start by creating a new database and add a versioning table to it, where we keep track of the database version.

# Will create a new database and add table to manage versions
Function Build-Database ([string]$sqlServer, [string]$databaseName)
{
	# http://sqlblog.com/blogs/allen_white/archive/2008/04/28/create-database-from-powershell.aspx
	[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO')  | out-null
	$s = new-object ('Microsoft.SqlServer.Management.Smo.Server') $sqlServer
	$dbname = $databaseName
	
	# Instantiate the database object and create database
	$db = new-object ('Microsoft.SqlServer.Management.Smo.Database') ($s, $dbname)
	$db.Create()
	
	# Create table and column for handling database version
	$db.ExecuteNonQuery("CREATE TABLE [$databaseName].[dbo].[Settings] ([DatabaseVersion] int NOT NULL)");
	$db.ExecuteNonQuery("INSERT INTO [$databaseName].[dbo].[Settings] ([DatabaseVersion]) VALUES (0)");
}

And now we can look through the directory that contains the change scripts and execute them on the database, one by one.

# Will update the database to the most current version in the database directory
Function Update-Database ([string]$connection_string, [string]$database_directory)
{

	$databaseVersion = Get-Database-Version $connection_string
	
	# Get all source files that have higher database version number
	$files = Get-ChildItem "$database_directory\*.sql" | Where { [int]::Parse($_.name.Substring(0, 4)) -gt $databaseVersion }
	
	# For each of those files, run query on database
	foreach ($file in $files)
	{
		$fileName = $file.name
		Write-Host "Apply update script: $fileName"
		
		# Get-Content returns a string array of all the lines. We join that into a single string
		$fileContents = Get-Content "$file"
		$sql = [string]::Join([Environment]::NewLine, $fileContents);
		Execute-Sql-Query $connectionString $sql
		
		# Get this version number
		$version = [int]::Parse($fileName.Substring(0, 4))
	
		# Update the settings database with current version number
		Execute-Sql-Query $connectionString "UPDATE [Settings] SET [DatabaseVersion] = $version"
	}
}

Last, there's some helper functions for the functions used above.

Function Get-Database-Version ([string]$connectionString)
{
	[System.Data.SqlClient.SqlConnection]::ClearAllPools()

	$sql = "SELECT TOP 1 [DatabaseVersion] FROM [Settings]"
	## Connect to the data source and open it
	$connection = New-Object System.Data.SqlClient.SqlConnection $connectionString
	$connection.Open()

	$command = New-Object System.Data.SqlClient.SqlCommand $sql,$connection
	$version = $command.ExecuteScalar();
	
	$connection.Close()
	$version
}

Function Validate-Connection ([string]$connectionString)
{
	[System.Data.SqlClient.SqlConnection]::ClearAllPools()
	
	try	{
		## Connect to the data source and open it
		$connection = New-Object System.Data.SqlClient.SqlConnection $connectionString
		$connection.Open()
		$connection.Close()
		$TRUE
	}
	catch {
		$FALSE
	}
}

Function Execute-Sql-Query ([string]$connectionString, [string]$sql)
{
	[System.Data.SqlClient.SqlConnection]::ClearAllPools()
	
	## Connect to the data source and open it
	$connection = New-Object System.Data.SqlClient.SqlConnection $connectionString
	$connection.Open()

	$server = New-Object Microsoft.SqlServer.Management.Smo.Server($connection)
	$server.ConnectionContext.ExecuteNonQuery($sql) | out-null
	
	$connection.Close()
}

Thanks to Gary Murphy for his contribution to Execute-Sql-Query, and getting it to work properly with GO-statements. You will find his blog at http://garyjmurphy.com/.

The whole script can be found in its repository on bitbucket.

Introducing Alive

Sun, 04 Dec 2011 11:19:15 GMT

Alive is a performance counter monitor for ASP.NET. You install it in an IIS application and can view it from anywhere. It's free, it's open source and it's happy pappy.

Prerequesits

Alive should work on any of the following.

IIS 6.0, IIS 7.0, IIS 7.5 and IIS Express
CLR 2.0, CLR 4
Web browser client: Firefox, Chrome (not IE)

Installation

There are several ways to install Alive described below. After the installation you should go to the Alive url.

http://yoursite/Alive.axd/

Project NuGet Installation

The NuGet package is designed to work well with Cassini, the build debug webserver in Visual Studio. That is why the NuGet package version is compiled for x86. This will probably not work well on your server, as most servers today is x64, but it was a design decision based on most developers are using Cassini as their primary web server during development.

http://nuget.org/List/Packages/LiteMediaAlive

PM> Install-Package LiteMediaAlive

Server installation

Go download the appropriate package from

http://code.google.com/p/litemedia-alive/downloads/list

The architecture x86/x64 has to match your server, and CLR version must match the application pool.

Copy Alive.dll into your bin directory.

Alive works through an HttpHandler. This means that you need to add a reference to the handler in your web.config.

IIS 6.0

<!-- IIS 6.0 Configuration -->
<system.web>
  <httpHandlers>
	<add path="Alive.axd" verb="*" type="LiteMedia.Alive.Handler, Alive"/>
  </httpHandlers>
</system.web>

IIS 7.0+

<!-- IIS 7+ Configuration -->
<system.webServer>
  <handlers>
	<add name="Alive" path="Alive.axd" verb="*" type="LiteMedia.Alive.Handler, Alive"/>
  </handlers>
</system.webServer>

Authorize your web application user to read performance counters

Before Alive can read performance counters off your system you need to grant it access. This means that

You need to run your application pool as a custom user
You need to add that user to the group "Performance Monitor Users" on the local machine.

Installation scripts will be added to the project when I got the time.

Configuration

Different installations of Windows might have different performance counters. Here's a default configuration to start from.

<configuration>
	<configSections>
		<sectionGroup name="alive" type="LiteMedia.Alive.Configuration, Alive">
			<section name="settings" type="LiteMedia.Alive.SettingsSection, Alive"/>
			<section name="counters" type="LiteMedia.Alive.CountersSection, Alive"/>
		</sectionGroup>
	</configSections>	
	
  <alive>
    <settings columns="3" />
    <counters>
      <groups>
        <group name="Hardware" updateLatency="1000">
          <counter name="CPU" categoryName="Processor" counterName="% Processor Time" instanceName="_Total" />
          <counter name="Memory" categoryName="Memory" counterName="Pages/sec" />
        </group>
        <group name="Memory" updateLatency="5000">
          <counter name="RAM" categoryName="Memory" counterName="% Committed Bytes In Use" />
          <counter name="Page file" categoryName="Paging File" counterName="% Usage" instanceName="_Total" />
        </group>
        <group name="ASP.NET Performance" updateLatency="1000">
		  <counter name="Queued req." categoryName="ASP.NET" counterName="Requests Queued" />
		  <counter name="Rejected req." categoryName="ASP.NET" counterName="Requests Rejected" />
          <counter name="Requests/sec" categoryName="ASP.NET Applications" counterName="Requests/Sec" instanceName="__Total__" />
        </group>
		<group name="IIS" updateLatency="5000">
		  <counter name="App Restarts" categoryName="ASP.NET" counterName="Application Restarts" />
		  <counter name="Recycles" categoryName="ASP.NET" counterName="Worker Process Restarts" />
		</group>
		<group name="Errors" updateLatency="5000">
		  <counter name="ASP.NET" categoryName="ASP.NET Applications" counterName="Errors Total" instanceName="__Total__" />
		</group>
		<group name="Session state server" updateLatency="5000">
		  <counter name="Active" categoryName="ASP.NET" counterName="State Server Sessions Active" />
		</group>
      </groups>
    </counters>
  </alive>
</configuration>

Every group element is a chart. You can place several counters in each chart. You specify the counters with the following attributes

name = name of the counter in the chart
categoryName = name of the performance counter category, called "Object" in the image below
counterName = name of the performance counter, called "Counter" in the image below
(optional) instanceName = name of the specific instance of this counter, called "Instance" in the image below

Easiest way to find performance counters is through the performance monitor in Windows. Just type "perfmon" in Start/Run and you will get the following interface.

Troubleshooting

Most common problem is that no counters are showing. When that happens, open up perfmon on the servern and verify that the counters aren't actually zero on the server. Now, this problem might relate to any of the following

You're not running under full trust. Performance counters won't work under medium trust, period.
Your IIS application pool identity is not a member of group "Performance Monitor Users" on the local machine.
Your machine architecture x86/x64 does not match the compiled DLL Alive.dll. Exchange it with the correct one from here
http://code.google.com/p/litemedia-alive/downloads/list

If you've verified all these, you need to turn on application logging.

Copy Common.Logging.dll. Common.Logging.Log4Net.dll and Log4Net.dll to your bin directory. Edit your web.config with the following additions.

<configuration>
  <configSections>
    <sectionGroup name="common">
      <section name="logging" type="Common.Logging.ConfigurationSectionHandler, Common.Logging" />
    </sectionGroup>
    <section name="log4net" type="log4net.Config.Log4NetConfigurationSectionHandler, log4net"/>
  </configSections>
  
  <common>
    <logging>
      <factoryAdapter type="Common.Logging.Log4Net.Log4NetLoggerFactoryAdapter, Common.Logging.Log4net">
        <arg key="configType" value="INLINE" />
      </factoryAdapter>
    </logging>
  </common>

  <log4net>
    <appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender">
      <file value="logs\log.txt" />
      <appendToFile value="true" />
      <rollingStyle value="Size" />
      <maximumFileSize value="1MB" />
      <maxSizeRollBackups value="10" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%date %-5level %logger - %message%newline" />
      </layout>
    </appender>

    <appender name="AuditFileAppender" type="log4net.Appender.RollingFileAppender">
      <file value="logs\activity.txt" />
      <appendToFile value="true" />
      <rollingStyle value="Size" />
      <maximumFileSize value="1MB" />
      <maxSizeRollBackups value="10" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%date %-5level %logger - %message%newline" />
      </layout>
    </appender>

    <logger name="alive-debug">
      <level value="ALL" />
      <appender-ref ref="LogFileAppender" />
    </logger>
    <logger name="alive-activity">
      <level value="ALL" />
      <appender-ref ref="AuditFileAppender" />
    </logger>
  </log4net>
</configuration>

Credits

Alive is open source (Apache License) hosted on Google Code. It was created by me, Mikael Lundin, and written in F# server side and CoffeScript on the client. Comments and bug reports are more than welcome.

Application logging for F#

Fri, 02 Dec 2011 05:46:27 GMT

Using existing log frameworks with F# is easy enough. Here's an example with log4net.

let log = log4net.LogManager.GetLogger("litemedia")
log.DebugFormat("Hello {0}, you are {1} years old", [|"Mikael", 30|])

Not very F#-ish now, is it. Looks like I've written C# in F# syntax. I would like to have a more Printf style of my logging messages.

Log.debug "Hello %s, you are %i years old" "Mikael" 30

We can easily accomplish this by wrapping the logging functionality in a module.

module Log =
  let private _log = log4net.LogManager.GetLogger("litemedia")
  let debug format = Printf.ksprintf _log.Debug format

ksprintf sends the result of sprintf into the _log.Debug method. This is pretty simple stuff, but useful when you know how.

Debug SOAP request and response in WCF

Fri, 04 Nov 2011 06:13:11 GMT

Often while working with SOAP service integrations I would like to see the actual SOAP envelope that is passed around. This is not trivial while working with WCF, because it effectivly hide the implementation details. You'll will have to turn on WCF message debugging.

That is done in web.config.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <system.diagnostics>
    <sources>
      <source name="System.ServiceModel.MessageLogging">
        <listeners>
          <add name="xml"/>
        </listeners>
      </source>
    </sources>
    <sharedListeners>
      <add name="xml" type="System.Diagnostics.XmlWriterTraceListener" initializeData="Traces.svclog" />
    </sharedListeners>
  </system.diagnostics>

  <system.serviceModel>
    <diagnostics wmiProviderEnabled="true">
      <messageLogging
           logEntireMessage="true"
           logMalformedMessages="true"
           logMessagesAtServiceLevel="true"
           logMessagesAtTransportLevel="true"
           maxMessagesToLog="3000"
       />
    </diagnostics>
  </system.serviceModel>
</configuration>

When you run your code a file called Traces.svclog should be created in your output directory. In a website this would be root of the site, and in a native application, this would be the bin folder.

You can open it in Microsoft Service Trace Viewer or just an ordinary text editor.

I find this very useful when dealing with services from other companies, where as I can really say - I'm sending this SOAP request and I'm recieving this response. That's not expected, is it?

If you're having trouble getting this to work, you may look at my sample application, that fetches the currency conversion rate from an open webservice at http://www.webservicex.net.

There were build errors. Would you like to continue and run the last successful build?

Thu, 03 Nov 2011 16:58:22 GMT

No.

Let me say that again.

No.

I can't believe how many times I've pushed "No" on this dialog. I really can't remember any time that I've pressed 'Yes'. Why is it even there? Is there somone that chooses to press 'Yes' on in this dialog? If you've changed something in the source code, and try to start the site in Debug mode, wouldn't you be interested in the changes? Wouldn't the ability to test the changes be the sole reason for you to start this site in Debug mode?

I learned recently that you can turn this dialog off, globally in Visual Studio, and not just the solution that you're in.

Go to menu Tools/Options
Project and Solutions / Build and Run
On Run, when build or deployment errors occur
=> Select Do not launch

Away with you, evil stupid dialog.

Git productive in TFS

Sun, 30 Oct 2011 11:41:03 GMT

Have you ever used TFS (Microsoft Team Foundation System) in a day to day basis? I'm sure its all fine and dandy for managers, but as a developer the revision control system of TFS gives me headaches. One or two reasons for this are...

Workspaces, where TFS will make all files readonly until you 'touch' them in Visual Studio 2010, trigger a 'check out for edit' which make your files writable. Works fine except...
TFS makes working in Visual Studio terrible slow. When you want to change a file it must first be checked out for edit by TFS (syncronously), and this does not happen on save - but on actual edit.
You accidently check out files for edit when you didn't intend to edit the file, or save your changes.
Edit files outside Visual Studio means manually doing a 'check out for edit' in the Source Control Manager to get rid of the readonly attribute to files.
The readonly attribute also has a way of finding itself in build servers, and production environment after copy operations. This can be a pain during continuous integration.
Sometimes you find yourself working offline, but you're not able to commit your changes without connection to your TFS. That's ok, but if you forget to press "Go Offline" before you loose connection, you will have Visual Studio crashing all over the place. Imagine working over cellular network where the connection comes and goes - it's impossible with TFS.

Nice to get that off my chest.

But TFS11 will solve those problems?

Yes, you will be able to have local workspaces in TFS11, but this will not be a DVCS as you will not be able to commit changes or merge branches while offline. All the work you'll do in your local workspace has to be synced with the TFS from time to time. I can imagine the headaches of merging local workspace with online workspace after a long period of offline work.

How to get going with git

The solution to the TFS problem is not using it for revision control, but only as centralized storage for Git. Here I will describe how to install git and explain my workflow with git and tfs.

Installation

Head over to the git homepage and install git itself. This will give you the binaries that you need to use git.

http://git-scm.com/

You need to install the Git-Tfs gateway in order to push and pull commits between Git and Tfs.

https://github.com/spraints/git-tfs/

The latest version at the time of writing this blog post is 0.12, and it has a defect of not being able to handle files with filenames not in ASCII. If you need to handle filenames with characters not in ASCII you should instead download the code from github and compile head version yourself. They've fixed it in there. Any version of Git-Tfs after 0.12 should have this problem fixed.

In order to get git to recognize the git-tfs extension they both need to be in the global environment path. The following figure explains where to find and edit this path variable.

I've added this value to the variable.

C:\Dev\Console2;C:\cygwin\bin;C:\Program Files\TortoiseSVN\bin;C:\Program Files (x86)\Git\bin;C:\Program Files (x86)\GitTfs

I use Cygwin and Console2 to enlighten my git experience.

For those that find console applications intimidating, should try the graphical tool called TortoiseGit. I've not tried it myself, but heard from colleages that it gives you an almost as good experience as working directly with the console.

http://code.google.com/p/tortoisegit/

Workflow

Working with git is not the same as working with TFS. You need to rethink the way you normaly do version control. Git is about versioning your code. Tfs is as much about centralizing your code, storing it all in one place.

When I was working with TFS, I would check in my work when I was done with a feature. Now when I work with Git, I will commit every change I do to the source code. I still push my completed user stories to TFS when they are done, so the frequency of my TFS checkins are not depleted, but I commit to git for every little change, giving me a complete history of 'fixing that bug' or 'completing that function'.

First you clone your TFS repository.

git tfs clone http://tfsserver:8080/DefaultCollection/ $/Path/In/Source/Control ProjectPath

This will give you a complete copy of the whole TFS history for that TFS path into ProjectPath. If you have a lot of history, and a lot of files, you need to be patient while all this is downloading. Thankfully you'll only have to do this once.

STOP! Yes, you have the code, but don't start coding just yet. You have a git repository and a master branch. The thing is, you never code directly into master branch. Instead you create a branch for every user story or bug you need to fix. Do it like this.

git checkout –b my_new_feature

This will create a new branch called "my_new_feature" and make it active. Now you can start development of your feature.

The three most important commands while developing your feature are status, add and commit.

git status = display what has changed
git add = stage changes for commit
git commit = commit staged changes

Let's commit a change to our new branch.

git commit -a -m "Test: Should do awesome when condition is satisfied"

What did happen during development of my_new_feature is that a coworker checked in his changes to TFS. Now we want to pull them out and update our branch with his changes. We du this with a tfs pull and rebase.

git checkout master
git tfs pull
git checkout my_new_feature
git rebase master

What happens when you rebase a branch is that you rewind all commits, rebase the branch on the new version in master, and then reapply your commits to that branch. You will get conflicts if you have made commits to the same stuff as what you pulled from tfs. A great aspect of this workflow is that the conflict will arise in a local branch where it won't affect other development if you would need to create a new branch before solving the conflict.

When you're done with your new feature, you should merge it to master and from there we could push it back into TFS, which is effectivly a TFS checkin.

git checkout master
git merge my_new_feature
git tfs checkintool --build-default-comment

What are the benefits of working with git?

This workflow seems to give you a lot of overhead. Should it be this hard to just manage versioning of software. Do you really get back all the time you invest in creating branches and merging back and forth?

You always have a clean master. When the customer comes to you with "drop everything you're doing, we need to do 'this' instead", you'll actually be able to do just that.
You're actually doing versioning, and not 'checkin at end of the day to keep a backup of your code' as I've seen many developers treat TFS
You work completely offline and may still do versioning. This has been a real issue for me while developing on the commute train.
You can branch, commit and do versioning without disturbing build server or colleages with your checkins.

Using git with TFS is probably the closest you can get, having a decent experience with TFS version source control, even if that means taking away 'version' and 'control' from the Team Foundation System.

Using TDD to test file system operations

Sun, 30 Oct 2011 11:41:03 GMT

I will refer to SUT during this article, meaning "System Under Test". In this case it will be the synchronize functionality in the FileSystemSynchronizer class.

A friend of mine asked me how I would unit test file system operations. The answer is that System.IO is not very test friendy and you will have to implement wrappers around it. This is not very hard, only time consuming and it gives you an additional layer of complexity. I would do this if I had to do a lot of file operations that needed testing, or if those file operations where important enough.

First we need a purpose. Let's say that we're going to build an application that's going to synchronize files in two file folders by pushing changes from source to target.

public class FileSystemSynchronizer
{
    public void Synchronize(string source, string destination)
    {
    }
}

Before we do anything else we should ask ourselves what we need this routine to accomplish. We do this by defining tests.

public class FileSystemSynchronizerSpecification
{
    [Fact]
    public void ShouldCreateTargetFolderIfDoesNotExist()
    {
    }

    [Fact]
    public void ShouldCopyAnyFilesFromSourceFolderToTargetFolder()
    {
    }

    [Fact]
    public void ShouldCopyAnyDirectoriesFromSourceFolderToTargetFolder()
    {
    }

    [Fact]
    public void ShouldCopyFilesAndFoldersRecursivlyFromSourceToTargetFolder()
    {
    }

    [Fact]
    public void ShouldNotCopySourceFileIfSameFileExistInTargetFolder()
    {
    }

    [Fact]
    public void ShouldCopySourceFileIfNewerThanFileInTargetFolder()
    {
    }

    [Fact]
    public void ShouldRemoveFilesFromTargetFolderNotPresentInSourceFolder()
    {
    }
}

Now we could just implement these tests and the wrapper situation should resolve itself?

[Fact]
public void ShouldCreateTargetFolderIfDoesNotExist()
{
    const string SourcePath = @"C:\FolderA";
    const string TargetPath = @"C:\FolderB";

    /* Setup */
    var factory = MockRepository.GenerateMock<IFileSystemFactory>();
    var sut = new FileSystemSynchronizer(factory);

    /* Arrange */
    factory.Expect(f => f.PathExists(TargetPath)).Return(false);

    /* Test */
    sut.Synchronize(SourcePath, TargetPath);

    /* Assert */
    factory.AssertWasCalled(f => f.MakeDirectory(TargetPath));
}

Do you find it hard to write tests before writing the code? Well, it is meant to be hard, because you should consider why you write the code in the first place.

I've created something I call the IFileSystemFactory and that class knows how to check if a path exists and it knows how to create directories. In this test I expect PathExists to be called and return false from it, and then I verify that a directory is created.

As you run this test it will turn red, but as you write the implementation it should turn green.

public class FileSystemSynchronizer
{
    private readonly IFileSystemFactory fsFactory;

    public FileSystemSynchronizer(IFileSystemFactory fsFactory)
    {
        this.fsFactory = fsFactory;
    }

    public void Synchronize(string sourcePath, string targetPath)
    {
        if (!fsFactory.PathExists(targetPath))
        {
            fsFactory.MakeDirectory(targetPath);
        }
    }
}

The last step is to refactor, but I think I will wait until I have something to refactor. This code is still quite simple and clean.

Let's implement that next test.

[Fact]
public void ShouldCopyAnyFilesFromSourceFolderToTargetFolder()
{
    /* Setup */
    var factory = MockRepository.GenerateMock<IFileSystemFactory>();
    var sut = new FileSystemSynchronizer(factory);

    var targetDirectory = MockRepository.GenerateStub<IDirectory>();

    var file1 = MockRepository.GenerateMock<IFile>();
    file1.Name = "first.txt";

    var file2 = MockRepository.GenerateMock<IFile>();
    file2.Name = "second.txt";

    var fileList = new[] { file1, file2 };

    /* Arrange */
    factory.Expect(f => f.PathExists(TargetPath)).Return(true);
    factory.Expect(f => f.GetDirectory(SourcePath)).Return(targetDirectory);
    targetDirectory.Files = fileList;

    /* Test */
    sut.Synchronize(SourcePath, TargetPath);

    /* Assert */
    foreach (var file in fileList)
    {
        file.AssertWasCalled(f => f.CopyTo(TargetPath));
    }
}

And the SUT to make it green.

public class FileSystemSynchronizer
{
    private readonly IFileSystemFactory fsFactory;

    public FileSystemSynchronizer(IFileSystemFactory fsFactory)
    {
        this.fsFactory = fsFactory;
    }

    public void Synchronize(string sourcePath, string targetPath)
    {
        if (!fsFactory.PathExists(targetPath))
        {
            fsFactory.MakeDirectory(targetPath);
        }

        foreach (var file in fsFactory.GetDirectory(sourcePath).Files)
        {
            file.CopyTo(targetPath);
        }
    }
}

And we're green, but .... THIS IS CRAP!

Overspecification is the hell of unit testing

What I did just now was not testing the function, but specifying the internals of the function. This is very dangerous, because I can't refactor without changing my tests. You should always try to test only the public api, and you should not bother with the internals. Instead you should look at the output after SUT has been run.

That means that we'll have to rethink and refactor our tests.

Let's create a virtual simulation of our file system instead, and fill it with files. Our virtual directory as source should be replicated into our virtual target path. This means a bit more implementation in the test, but we can limit the testing to Input/Output without overspecifying internals.

public class VirtualFileSystemFactory : IFileSystemFactory
{
    public static readonly IDirectory FolderA;
    public static readonly IDirectory FolderB;

    private const string FolderAPath = @"C:\FolderA";
    private const string FolderBPath = @"C:\FolderB";

    private readonly IDictionary<string, IDirectory> fileSystem = new Dictionary<string, IDirectory>
        {
            { FolderAPath, FolderA },
            { FolderBPath, FolderB },
        };

    static VirtualFileSystemFactory()
    {
        FolderA = new VirtualFolder(FolderAPath);
        FolderB = new VirtualFolder(FolderBPath);
    }

    public bool PathExists(string targetPath)
    {
        return fileSystem.ContainsKey(targetPath);
    }

    public void MakeDirectory(string targetPath)
    {
        fileSystem.Add(targetPath, new VirtualFolder(targetPath));
    }

    public IDirectory GetDirectory(string sourcePath)
    {
        if (!this.PathExists(sourcePath))
        {
            throw new DirectoryNotFoundException("Folder was not registered in VirtualFileSystemFactory: " + sourcePath);
        }

        return fileSystem[sourcePath];
    }

    public void Copy(IFile file, string targetPath)
    {
        this.GetDirectory(targetPath).Add(file);
    }
}

public class VirtualFile : IFile
{
    public VirtualFile(string name)
    {
        Name = name;
    }

    public string Name { get; set; }
}

public class VirtualFolder : IDirectory, IEnumerable<IFileSystemItem>
{
    private readonly string path;
    private readonly IList<IFileSystemItem> items;

    public VirtualFolder(string path)
    {
        this.path = path;
        items = new List<IFileSystemItem>();
    }

    public IEnumerable<IFile> Files
    {
        get { return items.Where(f => f is IFile).Cast<IFile>(); }
    }

    public void Add(IFile file)
    {
        items.Add(file);
    }

    public IEnumerator<IFileSystemItem> GetEnumerator()
    {
        return items.GetEnumerator();
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return items.GetEnumerator();
    }
}

That is a lot of code, but it is testing code. This will actually give us the power to not overspecify our tests, but work with the results of the method we're testing.

Look how this beautified the tests that where previously a mocking hell.

[Fact]
public void ShouldCreateTargetFolderIfDoesNotExist()
{
    const string UnknownTargetPath = @"C:\FolderC";

    /* Setup */
    var factory = new VirtualFileSystemFactory();
    var sut = new FileSystemSynchronizer(factory);

    /* Test */
    sut.Synchronize(SourcePath, UnknownTargetPath);

    /* Assert */
    Assert.True(factory.PathExists(UnknownTargetPath), "Target path should be created if it does not exist");
}

[Fact]
public void ShouldCopyAnyFilesFromSourceFolderToTargetFolder()
{
    /* Setup */
    var factory = new VirtualFileSystemFactory();
    var sut = new FileSystemSynchronizer(factory);

    // Create files in source folder
    var sourceFolder = factory.GetDirectory(SourcePath);
    sourceFolder.Add(new VirtualFile("first.txt"));
    sourceFolder.Add(new VirtualFile("second.txt"));

    /* Test */
    sut.Synchronize(SourcePath, TargetPath);

    /* Assert */
    var targetFolder = factory.GetDirectory(TargetPath);
    foreach (var file in sourceFolder.Files)
    {
        Assert.Contains(file, targetFolder.Files);
    }
}

These tests are great, because they won't break when we refactor our SUT. They are great because they are readable and you don't have to be Ayende to figure out how the mocking works.

What about wrapping the file system?

If I finish writing my tests and implementing my system, I will end up with a file system wrapping that looks like this.

We did wrap the file system. The wrapping layer grew fourth from what my tests needed. This means that it would probably not look the same if we had a different problem to solve. Then the wrapping layer would be suited for that problem instead.

Originating from the problem description and let the API grow from our tests, gave us a wrapping layer that both looks and feels natural to the problem at hand. I could never have anticipated this design, it has to be hand grown and it has to be done with TDD.

You can download the complete sample from here. Do I need to mention that it worked flawless on the first run?

Coverage is a broken metric

Sun, 30 Oct 2011 11:41:03 GMT

In the backwaters of previous blog entry, Using TDD to test file system operations, I've been thinking alot about test coverage. This is the metric you use to measure how many lines of code % you traverse with your tests.

This metric is meant to tell you

How much of the application you've tested
If you need to write more tests
When you're done testing

Unfortunatly it does not do these things.

Our dear Uncle Bob propagates 100% coverage at all times. This is not very sane. Some tests are just useless. Take my previous blog post as an example. What would it gain us to test the following implementation of the file system wrapper?

public bool PathExists(string targetPath)
{
    return Directory.Exists(targetPath);
}

In a matter of tests "driving the design", we're already done with that part. In a matter of regression testing, don't even go there. The only thing you would test here is that System.IO is doing what it is supposed to, and that is not your job. That is up to Microsoft to verify.

Every test you write must give you more value, compared to the cost of writing the test. This means that doing focused testing on the following things is just creating waste.

Wrapper classes
Constructors
Generated proxy classes (WCF services)
Trivial property setters and getters

If you accidently traverse these while running specifications that's fine. That takes us on to the real topic here.

What to test?

You should not test how your application operates, but confirm that it does what it is supposed to.

Test the 'what' not the 'how'.

Why? Because if you focus on testing 'what', your tests won't break when you refactor your code, because you changed how to do things. As long as your program behavior stays the same your tests will stay green. This is incredible valuable as test maintenance is an incredible waste. Your tests should do two things

Drive the design
Verify system behavior

Everything else is a waste.

Testing system behavior

This means that we don't have tests called "CalculatorTest" or "NamePropertyShouldGetSameValueAsWasSet" because this does not test system behavior. Instead our tests looks like this.

CreateTargetFolderIfDoesNotExist
CopyAnyFilesFromSourceFolderToTargetFolder
CopyAnyDirectoriesFromSourceFolderToTargetFolder
CopyFilesAndFoldersRecursivlyFromSourceToTargetFolder
NotCopySourceFileIfSameFileExistInTargetFolder
CopySourceFileIfNewerThanFileInTargetFolder
RemoveFilesFromTargetFolderNotPresentInSourceFolder

All in all I have a coverage of 34%. Why? Because I have a wrapping layer that enables me to test the system without hitting the file system. I'm not testing that the wrapping layer is correclty mapped against System.IO, because that is not a part of the system specification, it is just a side effect of testing and a waste to test.

Coverage does not prove the code to be correct

Complete coverage does not mean that you're done testing.

public double Divide(int a, int b)
{
    return a / b;
}

[Fact]
public void DivideTest()
{
    Assert.Equal(2, this.Divide(4, 2));
}

Yes, this gives you 100% coverage, but the question stands, is the code correct? Is Divide(5, 2) == 2 expected behavior? We write specifications to decide what behavior to expect from the system.

ShouldDivideNumbersWithAnIntegerResult
ShouldDivideNumbersWithFloatNumberResult
ShouldReturnZeroOnDivideWithZero

Two last specifications were not proven with our previous test and might cause unexpected behavior from our program. Implementing these specifications would mean that we have 300% coverage on Divide, but does that mean that it is fully covered? Still no.

Even Uncle Bob admits it. "100% code coverage does not mean your code is correct. It _does_ mean that you tried."

Conclusions about test coverage

Focus on verifying that your system does what it is supposed to do. Don't worry about coverage, it will not tell you if you're done testing. It can't tell you if you've proven your program to be correct and it will not tell you the quality of your tests.

Instead you should

Write a specification (red)
Implement that specification (green)
Refactor!

Infinite sequence of primes

Sun, 30 Oct 2011 11:41:02 GMT

I've been writing algorithms for calculating primes since high school. Most of those algorithms have looked the same, but in different languages with that language's specific quirks. It was when I read up on F# sequence expressions that my mind was blown, and I went holy bananas as I could see the solution to a prime calculation problem I've had since I started.

Is 15485867 a prime?

Yes, it is. How would I know that? I must decide that the number is not evenly divisible by any prime number up to the square root of that number (3935). Here comes the problem. I would have to know every prime up to 3935, to make sure that 15485867 is a prime.

For every number up to 3935 I have to divide by any prime up to the square root of that number. It looks like I would have to carry around a list of prime numbers in order to calculate any other prime number.

(you could also use the sieve of eratosthenes for getting primes up to a certain top limit)

Sequences

Instead I was going to use sequences. What is a sequence? It could be numbers where you would say "give me next" and you would get the next number until you reach the end. The thing about sequences is that you always move next. So when you want the third element, you would have to take next element three times.

What's so special about sequences is that they can be infinite. If you have a function that calculates the next element, the sequence does not have to have an end. If you calculate the sequence lazily, you would also be able to pass the infinite sequence around like any other object reference.

Sequence in C#

IEnumerable<T>

Sequence in F#

seq<'a>

Creating sequences

In F# you use sequence expressions to create sequences.

seq { for n in 0..100 do yield n }

This creates a simple sequence ranging from 0 to 100. You can do the same in C#.

Enumerable.Range(0, 100)

But sequence expressions are a bit more powerful than that. You can use yield! (bang!) to bind together sequences.

  seq { 
    yield 0
    yield! [1; 2; 3]
    yield! [4; 5; 6]
    yield! [7; 8; 9]
  }

So what about this?

  let rec alternating = 
    seq {
      yield true
      yield false
      yield! alternating
    }

val it : seq<bool> = seq [true; false; true; false; ...]

Not very surprising, this code generates an infinite sequence of alternating true/false. But why doesn't it stuck and loop forever generating the sequence? Because yield! is lazy and won't generate next true/false until you request it.

Infinitive sequence of primes

As I was reading about recursive sequence expressions my mind went, "could I create an infinite sequence of prime numbers?". It took me a good part of a day, but here's my solution.

let rec primes = 
  let isPrime number primes =
    let sqrtn = float >> sqrt >> int
    primes
  	  |> Seq.takeWhile (fun n -> n <= (sqrtn number))
  	  |> Seq.exists (fun n -> number % n = 0)
	  |> not

  let rec primes' current =
    seq {
  	  if primes |> isPrime current then
	    yield current
	  yield! primes' (current + 2)
    }
  seq {
    yield 2
    yield! primes' 3
  } |> Seq.cache

Let's break it down. We have an outer and inner sequence.

  seq {
    yield 2
    yield! primes' 3
  } |> Seq.cache

The outer sequence yields the number 2 as the first prime and then calls inner function to generate primes 3 and up.

  let rec primes' current =
    seq {
  	  if primes |> isPrime current then
	    yield current
	  yield! primes' (current + 2)
    }

The inner function yields current number if it is prime. If it is not prime it will recurse by adding current number with 2. (next test is 5)

  let isPrime number primes =
    let sqrtn = float >> sqrt >> int
    primes
  	  |> Seq.takeWhile (fun n -> n <= (sqrtn number))
  	  |> Seq.exists (fun n -> number % n = 0)
	  |> not

The inner isPrime function is not very complicated. It takes number to be tested as argument, as well as a sequence of primes up to this number. If the number is not divisible with any prime up to square root of itself, this should also be a prime.

But hey! Where does that sequence of primes come from? As you can see on line 11, the sequence of primes are actually a recursive call to the sequence we're about to create.

Crazy talk

As long as we don't try to check primeness of the same number in the recursion as we try to test for isPrime, we won't get into an infinite recursion. We make sure of that by only divide current number up to the square root of that number.

This becomes extremely ineffective since we need to recalculate the sequence and every prime up to square root of the number on recursion. That is why I've added Seq.cache on line 18. This will cache already calculated primes in the sequence and return them directly. This makes the code, not only beautiful to look at, but also pretty fast.

Is 15485867 a prime?

It is actually the 1 millionth and 1 prime.

primes |> Seq.nth(1000000)

(zero based indexer)

Dependency Injection in ASP.NET WebForms

Sun, 30 Oct 2011 11:41:02 GMT

To me, any webforms solution is legacy code. Sadly, most of my consultancy happens to be maintenance of these webforms behemoths. Even so, that doesn't stop me from trying to improve my own work process and the applications that I'm working with.

One huge improvement have been dependency injection into webforms page object. This has made the asp pages much easier to unit test. Here's a small guide on how to do dependency injection in a legacy webforms application with Unity.

Unity HttpModule

The trick is to hook into ASP.NET life cycle when the page tree structure is built up. We can do this with an HttpModule.

public class UnityHttpModule : IHttpModule
{
	private const string NamespacePrefix = "DIWebFormsExample";
	
	public void Init(HttpApplication context)
	{
		context.PreRequestHandlerExecute += OnPreRequestHandlerExecute;
	}

	public void Dispose()
	{
		// Nothing to dispose
	}

	// Get the controls in the page's control tree excluding the page itself
	private static IEnumerable GetControlTree(Control root)
	{
		foreach (Control child in root.Controls)
		{
			yield return child;
			foreach (Control c in GetControlTree(child))
			{
				yield return c;
			}
		}
	}

	private static void OnPreRequestHandlerExecute(object sender, EventArgs e)
	{
		/* Static content */
		if (HttpContext.Current.Handler == null)
		{
			return;
		}

		var handler = HttpContext.Current.Handler;
		Unity.Instance.Container.BuildUp(handler.GetType(), handler);

		// User Controls are ready to be built up after the page initialization is complete
		var page = HttpContext.Current.Handler as Page;
		if (page != null)
		{
			page.InitComplete += OnPageInitComplete;
		}
	}

	// Build up each control in the page's control tree
	private static void OnPageInitComplete(object sender, EventArgs e)
	{
		var page = (Page)sender;
		foreach (Control c in GetControlTree(page))
		{
			var typeFullName = c.GetType().FullName ?? string.Empty;
			var baseTypeFullName = c.GetType().FullName ?? string.Empty;

			// Filter on namespace to avoid build up of System.Web components
			if (typeFullName.StartsWith(NamespacePrefix) || 
				baseTypeFullName.StartsWith(NamespacePrefix))
			{
				Unity.Instance.Container.BuildUp(c.GetType(), c);
			}
		}
	}
}

Please notice that we define the namespace on line 3 as a filter, because we don't want to have Unity build up controls that belongs to the .NET Framework.

We register this module in web.config as following.

<configuration>
	<!-- IIS 6.0 and below -->
    <system.web>
      <httpModules>
        <add name="UnityHttpModule" type="DIWebFormsExample.Lib.UnityHttpModule, DIWebFormsExample.Lib"/>
      </httpModules>
    </system.web>

	<!-- IIS 7.0 and above -->
    <system.webServer>
      <modules>
        <remove name="UnityHttpModule"/>
        <add name="UnityHttpModule" type="DIWebFormsExample.Lib.UnityHttpModule, DIWebFormsExample.Lib"/>
      </modules>
    </system.webServer>
</configuration>

This simply allows us to inject our dependencies through property injection. Here's our product page.

<form id="form1" runat="server">
<div>
    <h1>Products</h1>
    <h2>Electronics</h2>
    <asp:DataGrid 
        runat="server" 
        DataSource='<%# GetByType("Electronics") %>' 
        AutoGenerateColumns="true" OnLoad="DataBind" />
    <h2>Candy</h2>
    <asp:DataGrid 
        runat="server" 
        DataSource='<%# GetByType("Candy") %>' 
        AutoGenerateColumns="true" OnLoad="DataBind" />
</div>
</form>

public partial class ProductList : System.Web.UI.Page
{
	[Dependency]
	public IRepository<Product> Repository { get; set; }

	protected void Page_Load(object sender, EventArgs e)
	{
	}

	protected IEnumerable<Product> GetByType(string type)
	{
		return Repository.Get(p => p.Type == type);
	}

	protected void DataBind(object sender, EventArgs e)
	{
		((Control) sender).DataBind();
	}
}

On line 4 we inject our dependency through property injection by adding the attribute [Dependency] to the property. Unity will then resolve this dependency during BuildUp. I have registered this to my container as following:

container.RegisterType<IRepository<Product>, ProductRepository>();

Testing

How does this relate to testing? It makes it much easier to unit test our webforms code behind.

[TestFixture]
public class ProductListCodeBehindShould
{
	private class Template : ProductList
	{
		public Template()
		{
			Repository = new StubRepository();
		}

		public new IEnumerable<Product> GetByType(string type)
		{
			return base.GetByType(type);
		}

		private class StubRepository : IRepository<Product>
		{
			public IEnumerable<Product> Get(Func<Product, bool> criteria)
			{
				return new[]
				{
					new Product("Stereo", 120.0, "Electronics"),
					new Product("Candy bar", 1.2, "Candy"),
					new Product("Soda", 1.8, "Candy"),
					new Product("Deep fried snickers bar", 0.2, "Candy")
				}.Where(criteria);
			}
		}
	}

	[TestCase("Electronics", 120.0)]
	[TestCase("Candy", 3.2)]
	public void GetProductsBySpecifiedType(string type, double totalPrice)
	{
		/* Setup */
		var template = new Template();

		/* Test */
		var products = template.GetByType(type);

		/* Assert */
		Assert.That(products.Sum(p => p.Price), Is.EqualTo(totalPrice));
	}
}

What happens here? I create an sub class to the ASP.NET page code behind that I want to test, because I want to reach protected methods like GetByType(string type). My test makes sure that type is properly translated into a correct lambda expression, without touching the real ProductRepository. This is exactly what we need in order to test our code behind properly.

Summary

What I like about this approach is...

It's unobtrusive. Most testing approaches require you to apply a pattern like MVP in order to reap benefits of testing your UI layer. This might require you to rewrite a large part of your application. My DI module will not affect old code, but enable testability of any new code you write. Later on, you may refactor old code with dependency injection to improve testability.

This means that the initial cost is very small.
It works also for ASCX (user controls) and ASHX (handlers).

Go download a full sample here.

Dealing with dependencies in functional programming

Sun, 30 Oct 2011 11:41:01 GMT

What is a dependency?

Some entity that your unit depends on. (too abstract description)
Something that you want to replace while unit testing (too concrete description)

In OO programming you will use a dependency injection framework to build dependency chains. You use it to inject dependencies into constructors, methods and parameters. That is all very cool and enables testability, but not very useful in functional programming.

Why do we need to think about dependencies in functional programming?

Testing your units are still important, and to enable that you need to stub away your dependencies
Coupling also exists in functional programming and should be considered an evil

Function argument dependency injection

The most obvious way to handle dependencies is to send them into the function as an argument.

// Model object
type Person = { Name : string; BirthDate : System.DateTime }

// age: get the age of a person from name
// string -> (string -> Person) -> int
let age name (fn_getPerson : string -> Person) =

    // Get birthdate from dependency
    let birthDate = (fn_getPerson name).BirthDate 

    // Calculate
    (DateTime.Now - birthDate).Days / 365

val age : string -> (string -> Person) -> int

The dependency is a function that knows how to get a person from the database by calling with its name. While unit testing we really want to get rid of that dependency and that is why we send it in as a function, instead of calling it directly from where we need its result.

age "Mikael Lundin" DataAccess.GetPerson
// or maybe
DataAccess.GetPerson |> age "Mikael Lundin"

Testing would look like this. Here I'm using xUnit and Unqoute for testing.

open Xunit
open Swensen.Unquote

[<Fact>]
let ``send dependency into the function as an argument`` () =
    // Setup: In our test we return a Person directly
    let dependency (s : string) = { Name = s; BirthDate = new DateTime(1982, 7, 15) }

    // Assert
    test <@ 29 = age "Mikael" dependency  @>

Unit testing with Xunit and Unquote is a blast. I love the expressiveness that Unquote gives us. If we where to run this with Resharper as testrunner, and fail the test - we get this great test failure.

Oh, I love it!

Partial function application injection

This is a very scary title, but another functional way to deal with dependencies. Instead of accepting the dependency as an argument, you return a function that have to be completed with the dependency to run.

// Dependency through partial function
let age name : ((string -> Person) -> int) =
    // Return a function
    (fun (getPerson : string -> Person) -> 
        // Get birthdate
        let birthDate = getPerson(name).BirthDate

        // Return age
        (DateTime.Now - birthDate).Days / 365
    )

val age : string -> (string -> Person) -> int

This is indeed a bit harder to read and makes the function signature a bit messy. I return a function that will accept the dependency as an argument. Anyway I do like this method better than the previous.

age "Mikael" DataAccess.GetPerson
// this also works
DataAccess.GetPerson |> age "Mikael"

And here's a test.

[<Fact>]
let ``using a partial function result to insert the dependency to`` () =
    // Setup: Our dependency is called on the function result
    let dependency s = { Name = s; BirthDate = new DateTime(1982, 7, 15) }

    // Assert
    test <@ 28 = (age "Mikael") dependency @>

Interface dependency injection

Since F# has object literals, it is very easy to stub out interfaces, and when talking to OO libraries you might have to resort to this method. Consider our dependency.

public interface IUserRepository
{
    IEnumerable<User> GetUsers();
}

public class User
{
    public string UserName { get; set; }

    public string Password { get; set; }
}

We would like to write an authenticate method in F#. How should we handle our dependency? Here's some F# equivalient of what we would do in OOP.

let authenticate username password (rep : IUserRepository) =
    rep.GetUsers() |> Seq.exists (fun u -> u.UserName = username && u.Password = password)

When calling this from a test, we can use object literals to change the implementation of the interface members.

[<Fact>]
let ``using an interface as dependency`` () =
    // GetUsers() result
    let users = [| new User(UserName = "mlu", Password = "secret") |]

    // Fake repository
    let repository = { 
        new IUserRepository with
            member x.GetUsers() = users |> Seq.ofArray
    }

    // Assert
    test <@ true = authenticate "mlu" "secret" repository @>

But a more functional approach would be, not to use the interface as a dependency, but focus on the actual function as we did on the first two patterns. Let the caller of this function worry about on what object instance the dependent function lives.

let authenticate username password : ((unit -> seq<User>) -> bool) =
    (fun (getUsers : unit -> seq<User>) ->
        getUsers() |> Seq.exists 
            (fun user -> user.UserName = username && user.Password = password)

val authenticate : string -> string -> (unit -> seq<User>) -> bool

(authenticate "mlu" "secret") (new UserRepository()).GetUsers

Orchard: Alternative content on start page

Sun, 30 Oct 2011 11:41:01 GMT

It took me a lot of time to figure out how to tell Orchard to render something different on the start page. One way is to define a layout that is specific for the start page url.

If you want to use placement.info to move things around or hide the title on start page you can match on the ~/ url.

<!-- HIDE TITLE ON START PAGE -->
<Place Parts_RoutableTitle="Content:5" />
<Match Path="~/">
  <Place Parts_RoutableTitle="" />
</Match>

This tells Orchard to display page title in Content, unless the page has url ~/. Then you should not display it at all. Hope this will help someone else out there looking for an answer to this.

About and Contact

Sun, 30 Oct 2011 11:41:01 GMT

Do you recognize the content on this site? That is because it was earlier called Mint and located at http://mint.litemedia.se. I decided to move the blog to its own domain, because it has such importance to me. I update it nearly every week and it deserves more attention. I also choose to move it away from .se, since all my blog posts is in English I don't want this blog to be associated as a Swedish blog.

My name is Mikael Lundin and I'm the author of the content here. Any problem solved is worth little more than nothing unless you choose to share the problem solution. That is why most of my posts here are problems that I run into in my day to day work and the solution to those problems. I will also share content that I create for seminars and research on new languages and development platforms.

As of now I'm a .NET developer tired of the .NET community. What that means for the future, I don't know but I'm not going to label this as a .NET blog anymore, as I did when it was located at mint.litemedia.se. We'll just have to wait and see where the most interesting system developing is at, and I will be found there.

Contact

If you have any questions or ideas, please contact me.

Extending types in F#

Sun, 30 Oct 2011 11:41:01 GMT

Just as you can extend existing types in C# with extension methods you can do the same in F# with type extensions. The difference is that extending types in F# helps you work in a functional matter with object orientation. Consider the following discriminated union that could be used for a card game.

type Suit = | Spades | Hearts | Diamonds | Clubs

type Card =
    | ValueCard of Suit * int
    | Knave of Suit
    | Queen of Suit
    | King of Suit
    | Ace of Suit

We would like to expose a hand as a type. In this type we can do all the object oriented behavior things like limiting number of cards in hand to 5, making sure that we don't have duplicate cards and so on. Great, but the problem is working with the type members we always need to reevaluate the type as we're changing member functions.

type Hand (cards : Card list) =
    member this.Cards = cards

So we keep the Hand type as simple as possible, and then we extend with functionality.

let isRoyalStraightFlush (cards : Card list) = 
    match cards with
    | Ace suit_1 :: King suit_2 :: Queen suit_3 :: Knave suit_4 :: ValueCard (suit_5, 10) :: [] 
        when suit_1 = suit_2 && suit_2 = suit_3 && suit_3 = suit_4 && suit_4 = suit_5 
        -> true
    | _ -> false

The function isRoyalStraightFlush can be written and tested individually without any connection to the Hand object, and then we can extend the hand with this functionality like this.

type Hand with
    member x.IsRoyalStraightFlush = isRoyalStraightFlush x.Cards

I really like this way of working with object orientation in a functional manner.