Normally you can’t broadly stop someone from being able to send you mail. However, there is a loophole.
You can file a PS Form 1500 and say that the advertisement you received from them made you horny. No questions asked prohibitory order.
Normally you can’t broadly stop someone from being able to send you mail. However, there is a loophole.
You can file a PS Form 1500 and say that the advertisement you received from them made you horny. No questions asked prohibitory order.
I just had to setup my vimrc
and vimfiles
on a new laptop for work, and had some fun with Vim, mostly as itās been years since I had to do it. I keep my vimfiles
folder in my github, so I can grab it wherever I need it.
To recap, one of the places that Vim will look for things is $HOME/vimfiles/vimrc
, where $HOME
is actually the same as %USERPROFILE%
. In most corporate environments, the %USERPROFILE%
is actually stored in a networked folder location, to enable roaming profile support and help when a user gets a new computer.
So you can put your vimfiles
there, but, itās a network folder - itās slow to start an instance of Vim. Especially if you have a few plugins.
Instead, what you can do is to edit the _vimrc
file in the Vim installation folder (usually in C:\Program Files (x86)\vim
), delete the entire contents and replace it with:
set rpt+=C:\path\to\your\vimfiles
set viminfo+=nC:\path\to\your\vimfiles\or\whatever
source C:\path\to\your\vimfiles\vimrc
What this does is:
vimrc
file and uses thatThis post largely serves as a memory aid for myself when I need to do this again in future I wonāt spend longer than I probably needed to googling it to find out how to do it, but I hope it helps someone else.
Recently I was inspired by @buhakmehās blog post, Supercharge Blogging With .NET and Ruby Frankenblog to write something similar, both as an exercise and excuse to blog about something, and as a way of tidying up the metadata on my existing blog posts and adding header images to old posts.
The initial high level requirements I want to support are:
The next series of posts will cover implementing the above requirements⦠not necessarily in that order. First I will go over setting up the project and configuring Oakton.
After that I will probably cover implementing fixes to the existing blog metadata, as I think that is going to be something that will be required in order for any sort of Info function to work properly, as all of the yaml metadata will need to be consistent.
Then I think Iāll tackle the image stuff, which should be fairly interesting, and should give a nice look to the existing posts, as having prominent images for posts is part of the theme for the blog, which Iāve not really taken full advantage of.
Iāll try to update this post with links to future posts, or else make it all a big series.
dotnet new console --name BlogHelper9000
At work, we have recently been porting our internal web framework into .net 6. Yes, we are late to the party on this, for reasons. Suffice it to say I currently work in an inherently risk averse industry.
Anyway, one part of the framework is responsible for getting reports from SSRS.
The way it did this is to use a wrapper class around a SOAP client generated from good old ReportService2005.asmx?wsdl
, using our faithful friend svcutil.exe
. The wrapper class used some TaskCompletionSource
magic on the events in the client to make the client.LoadReportAsync
and the other *Async
methods actually async, as the generated client was not truely async.
Fast forward to the modern times, and we need to upgrade it. How do we do that?
Obviously, Microsoft are a step ahead: svcutil
has a dotnet version - dotnet-svcutil
. We can install it and get going:
dotnet too install --global dotnet-svcutil
Once installed, we can call it against the endpoint:
dotnet-svcutil http://server/ReportServer/ReportService2005.asmx?wsdl
In our wrapper class, the initialisation of the client has to change slightly, because the generated client is different to the original svcutil
implementation. Looking at the diff between the two files, itās because the newer version of the client users more modern .net functionality.
The wrapper class constructor has to be changed slightly:
public Wrapper(string url, NetworkCredential credentials)
{
var binding = new BasicHttpBinding(BasicHttpSecurityMode.TransportCredentialOnly);
binding.Security.Transport.ClientCredentialType = HttpClientCredentialType.Ntlm;
binding.MaxReceivedMessageSize = 10485760; // this is a 10mb limit
var address = new EndpointAddress(url);
_client = new ReportExecutionServiceSoapClient(binding, address);
_client.ClientCredentials.Windows.AllowedInpersonationLevel = TokenImpersonationLevel.Impersonation;
_client.ClientCredentials.Windows.ClientCredential = credentials;
}
Then, the code which actually generates the report can be updated to remove all of the TaskCompletionSource
, which actually simplifies it a great deal:
public async Task<byte[]> RenderReport(string reportPath, string reportFormat, ParameterValue[] parameterValues)
{
await _client.LoadReportAsync(null, reportPath, null);
await _client.SetExecutionParametersAsync(null, null, parameterValues, "en-gb");
var deviceInfo = @"<DeviceInfo><Toolbar>False</ToolBar></DeviceInfo>";
var request = new RenderRequest(null, null, reportFormat, deviceInfo);
var response = await _client.RenderAsync(request);
return response.Result;
}
You can then do whatever you like with the byte[]
, like return it in an IActionResult
or load it into a MemoryStream
and write it to disk as the file.
who is eating cereal anymore? Literally don’t think I’ve seen someone eat a bowl of cereal in twenty years
Recently we realised that we had quite a few applications being deployed through Octopus Deploy, and that we had a number of Environments, and a number of Channels, and that managing the ports being used in Dev/QA/UAT across different servers/channels was becoming⦠problematic.
When looking at this problem, itās immediately clear that you need some way of dynamically allocating a port number on each deployment. This blog post from Paul Stovell shows the way, using a custom Powershell build step.
As weād lost track of what sites were using what ports, and that we also have ad-hoc websites in IIS that arenāt managed by Octopus Deploy, we thought that asking IIS āHey, what ports are the sites you know about using?ā might be a way forward. We also had the additional requirement that on some of our servers, we also might have some arbitary services also using a port and that we might bump into a situation where a port was chosen that was already being used by a non-IIS application/website.
Researching the first situation, itās quickly apparent that you can do this in Powershell, using the Webadministration
module. Based on the answers to this question on Stackoverflow, we came up with this:
Import-Module Webadministration
function Get-IIS-Used-Ports()
{
$Websites = Get-ChildItem IIS:\Sites
$ports = foreach($Site in $Websites)
{
$Binding = $Site.bindings
[string]$BindingInfo = $Binding.Collection
[string]$Port = $BindingInfo.SubString($BindingInfo.IndexOf(":")+1,$BindingInfo.LastIndexOf(":")-$BindingInfo.IndexOf(":")-1)
$Port -as [int]
}
return $ports
}
To get the list of ports on a machine that are not being used is also fairly straightforward in Powershell:
function Get-Free-Ports()
{
$availablePorts = @(49000-65000)
$usedPorts = @(Get-NetTCPConnection | Select -ExpandProperty LocalPort | Sort -Descending | Where { $_ -ge 49000})
$unusedPorts = foreach($possiblePort in $usedPorts)
{
$unused = $possiblePort -notin $usedPorts
if($unused)
{
$possiblePort
}
}
return $unusedPorts
}
With those two functions in hand, you can work out what free ports are available to be used as the ānext portā on a server. Itās worth pointing out that if a site in IIS is stopped, then IIS wonāt allow that port to be used in another website (in IIS), but the port also doesnāt show up as a used port in netstat -a
, which is kind of what Get-NetTCPConnection
does.
function Get-Next-Port()
{
$iisUsedPorts = Get-IIS-Used-Ports
$freePorts = Get-Free-Ports
$port = $freePorts | Where-Object { $iisUsedPorts -notcontains $_} | Sort-Object | Select-Object First 1
Set-OctopusVariable -Name "Port" -Value "$port"
}
Then you just have to call it at the end of the script:
Get-Next-Port
Youād also want to have various Write-Host
or other logging messages so that you get some useful output in the build step when youāre running it.
If you found this because you have a build server which is āofflineā, without any external internet access because of reasons, and you canāt get your build to work because dotnet fails to restore the tool you require for your build process because of said lack of external internet access, then this is for you.
In hindsight, this may be obvious for most people, but it wasnāt for me, so here it is.
In this situation, you just need to shy away from local tools completely, because as of yet, Iāve been unable to find anyway of telling dotnet not to try to restore them, and they fail every build.
Instead, Iāve installed the tool(s) as a global tool, in a specific folder, e.g. C:\dotnet-tools
, which Iāve then added to the system path on the server. You may need to restart the build server for it to pick up the changes to the environment variable.
One challenge that remains is how to ensure the dotnet tools are consistent on both the developer machine, and the build server. I leave that as an exercise for the reader.
Iām leaving this here so I can find it again easily.
We had a problem updating the Visual Studio 2019 Build Tools on a server, after updating an already existing offline layout.
I wonāt go into that here, because itās covered extensively on Microsoftās Documentation website.
The installation kept failing, even when using --noweb
. It turns out that when your server is completely cut off from the internet, as was the case here, you also need to pass --noUpdateInstaller
.
This is because (so it would seem) that even though --noweb
correctly tells the installer to use the offline cache, it doesnāt prevent the installer from trying to update itself, which will obviously fail in a totally disconnected environment.
Since a technical breakdown of how Betsy does texture compression was posted, I wanted to lay out how the compressors in Convection Texture Tools (CVTT) work, as well as provide some context of what CVTT's objectives are in the first place to explain some of the technical decisions.
First off, while I am very happy with how CVTT has turned out, and while it's definitely a production-quality texture compressor, providing the best compressor possible for a production environment has not been its primary goal. Its primary goal is to experiment with compression techniques to improve the state of the art, particularly finding inexpensive ways to hit high quality targets.
A common theme that wound up manifesting in most of CVTT's design is that encoding decisions are either guided by informed decisions, i.e. models that relate to the problem being solved, or are exhaustive. Very little of it is done by random or random-like searching. Much of what CVTT exists to experiment with is figuring out techniques which amount to making those informed decisions.
Anyway, CVTT's ParallelMath module is kind of the foundation that everything else is built on. Much of its design is motivated by SIMD instruction set quirks, and a desire to maintain compatibility with older instruction sets like SSE2 without sacrificing too much.
Part of that compatibility effort is that most of CVTT's ops use a UInt15 type. The reason for UInt15 is to handle architectures (like SSE2!) that don't support unsigned compares, min, or max, which means performing those operations on a 16-bit number requires flipping the high bit on both operands. For any number where we know the high bit is zero for both operands, that flip is unnecessary - and a huge number of operations in CVTT fit in 15 bits.
The compare flag types are basically vector booleans, where either all bits are 1 or all bits are 0 for a given lane - There's one type for 16-bit ints, and one for 32-bit floats, and they have to be converted since they're different widths. Those are combined with several utility functions, some of which, like SelectOrZero and NotConditionalSet, can elide a few operations.
The RoundForScope type is a nifty dual-use piece of code. SSE rounding modes are determined by the CSR register, not per-op, so RoundForScope when targeting SSE will set the CSR, and then reset it in its destructor. For other architectures, including the scalar target, the TYPE of the RoundForScope passed in is what determines the operation, so the same code works whether the rounding is per-op or per-scope.
While the ParallelMath architecture has been very resistant to bugs for the most part, where it has run into bugs, they've mostly been due to improper use of AnySet or AllSet - Cases where parallel code can behave improperly because lanes where the condition should exclude it are still executing, and need to be manually filtered out using conditionals.
by OneEightHundred (noreply@blogger.com) at 2021-01-03 23:21
If you want some highlights:
The SDL variant ("AerofoilSDL") is also basically done, with a new OpenGL ES 2 rendering backend and SDL sound backend for improved portability. The lead version on Windows still uses D3D11 and XAudio2 though.
Unfortunately, I'm still looking for someone to assist with the macOS port, which is made more difficult by the fact that Apple discontinued OpenGL, so I can't really provide a working renderer for it any more. (Aerofoil's renderer is actually slightly complicated, mostly due to postprocessing.)
In the meantime, the Android port is under way! The game is fully playable so far, most of the work has to do with redoing the UI for touchscreens. The in-game controls use corner taps for rubber bands and battery/helium, but it's a bit awkward if you're trying to use the battery while moving left due to the taps being on the same side of the screen.
Most of the cases where you NEED to use the battery, you're facing right, so this was kind of a tactical decision, but there are some screens (like "Grease is on TV") where it'd be really nice if it was more usable facing left.
I'm also adding a "source export" feature: The source code package will be bundled with the app, and you can just use the source export feature to save the source code to your documents directory. That is, once I figure out how to save to the documents directory, which is apparently very complicated...
Anyway, I'm working on getting this into the Google Play Store too. There might be some APKs posted to GitHub as pre-releases, but there may (if I can figure out how it works) be some Internal Testing releases via GPS. If you want to opt in to the GPS tests, shoot an e-mail to codedeposit.gps@gmail.com
Maybe, but there are two obstacles:
The game is GPL-licensed and there have reportedly been problems with Apple removing GPL-licensed apps from the App Store, and it may not be possible to comply with it. I've heard there is now a way to push apps to your personal device via Xcode with only an Apple ID, which might make satisfying some of the requirements easier, but I don't know.
Second, as with the macOS version, someone would need to do the port. I don't have a Mac, so I don't have Xcode, so I can't do it.
by OneEightHundred (noreply@blogger.com) at 2020-10-20 11:09
As part of modernisng, updating and generally overhauling my blog, I thought it would be nice to add some consistancy to the Yaml front matter used by Jekyll. For those who do not know, Jekyll uses Yaml front matter blocks to process any file which contains one as a special file. The front matter can contain variables in the form foo: value
. Jekyll itself defines some predefined globabl variables and variables for posts, but anything else is valid and can be use in Liquid tags.
I wondered if I could write some F# to:
Fairly straightforward requirements.
Iām using YamlDotNet to do most of the heavy lifting. I think could also have used the FSharp.Configuration Type Provider, but Iām not sure that it would have done exaclty what I wanted.
Iām just writing this in an F# script, hosted in a project. After adding the YamlDotNet NuGet package, we can reference it and get to work:
#r "../../.nuget/packages/YamlDotNet/8.1.2/lib/netstandard2.1/YamlDotNet.dll"
open System.IO
open System.Text.RegularExpressions
open YamlDotNet.Serialization
open YamlDotNet.Serialization.NamingConventions
let path = "../sgrassie.github.io/_posts"
Here, we reference the package, and then open various namespaces for use later on. The code for my blog is kept in a separate folder, relative to the project which has got the fsharp scripts Iām writing abot in it. This is nice and easy.
type FrontMatter() =
member val Title = "" with get, set
member val Description = "" with get, set
member val Layout = "" with get, set
member val Tags = [|""|] with get, set
member val Published = "" with get, set
member val Category = "" with get, set
member val Categories = "" with get, set
member val Metadescription = "" with get, set
member val Series = "" with get, set
member val Featured = false with get, set
member val Hidden = false with get, set
member val Image = "" with get, set
[<YamlMember(Alias = "featured_image", ApplyNamingConventions = false)>]
member val FeaturedImage = "" with get, set
[<YamlMember(Alias = "featured_image_thumbnail", ApplyNamingConventions = false)>]
member val FeaturedImageThumbnail = "" with get, set
[<YamlIgnore>]
member val MarkdownFilePath = "" with get, set
This is a class with auto-implemented properties. You can see three attributes in use. The YamlMember
attribute allows us to alias a property in Yaml which doesnāt follow the CamelCase convention we configured the deserialiser with. I think that a C# version of this would look pretty much the same.
let deserializer = DeserializerBuilder()
.WithNamingConvention(CamelCaseNamingConvention.Instance)
.Build()
This initialises the YamlDotNet deserialiser, and is pretty much almost exactly how you would do this in C#. To deserialise something, we need some Yaml. When I was testing this, I got an error in YamlDotNet that was pretty weird and essentially means that it canāt parse the file, and it turns out itās because all the other stuff outside the Yaml front matter that is upsetting it.
let expression = "(?:---)(?<yaml>[\\s\\S]*?)(?:---)"
Oh regex, I do love thee.
Very simply, this regex will parse everything in a file between two ---
blocks, into a named Yaml
group. We now have actual front matter, we still need to parse into an object.
let extractFrontmatter filePath =
let file = File.ReadAllText(filePath)
let result = Regex.Match(file, expression).Groups.["yaml"].Value
let frontMatter =
let frontMatter = deserializer.Deserialize<FrontMatter>(result)
frontMatter.MarkdownFilePath <- filePath
frontMatter
frontMatter
This is a bit more complex so lets unpack it:
filePath
.deserializer
, and return it. Here, we also keep track of the file path (we will need this later).We also need to load all of the markdown files:
let loadMarkdownFiles path = Directory.EnumerateFiles(path, "*.md", SearchOption.AllDirectories)
Notice how those last couple of functions are using ācurryingā. It lets us do all of the work in one pipeline:
path |> loadMarkdownFiles |> Seq.map extractFrontmatter |> Seq.iter (fun x -> printfn "%s - %s" x.MarkdownFilePath x.Title)
This gives us a dataset to work with. Next time weāll continue with the rest of the requirements.
Many years ago, after working in my first programming job for a couple of years the company was taken over, and coding tests for new hires were introduced. The incumbent developers all decided to take the test, and it was seen as a fun diversion for a couple of hours.
I donāt have access to the actual wording of the requirements given to candidates, but the test required a text file containing around 100k words to be loaded and sorted into the largest set of the longest anagram. For example in the words file Iām using in this blog post, there are 466544 words in the file, 406627 of which are anagrams. The largest set is for a 7 letter anagram, of whih there are 15 words. There are smaller sets of longer anagrams, weāre not interested in those. And, it had to run in in less a second. They had three hours to write it, on a computer not connected to the internet. They had access to Java, through Eclipse, C/C++/C# through Visual Studio and Delphi through Embarcadero Studio.
I donāt know where the test originally came from - I think it originated in a different company which had been acquired by the same company I now worked for, but Iām not sure. I think the intent of the test was to in part gauge how the candiate reacted to the deadline pressure, part how they could understand the requirements given to them, and lastly what sort of code they wrote.
As it has been a long time and the company no longer recruits after moving most development overseas, so, Iām going to present my solution.
First we have to load the file, and figure out to generate the anagram and keep track of how many instances of that anagram there are. It turned out for the candidates taking the test that this was the bit that most got stuck on, specifically the short mental leap it took to working out you needed to sort the letters of the word alphabetically to create the key.
private static string CreateKey(string word)
{
var lowerCharArray = word.ToLowerInvariant().ToCharArray();
Array.Sort(lowerCharArray);
return new string(lowerCharArray);
}
private static void LoadWords(string filePath, Dictionary<string, List<string>> words)
{
using (var streamReader = File.OpenText(filePath))
{
string s;
while ((s = streamReader.ReadLine()) != null)
{
var key = CreateKey(s);
if (words.TryGetValue(key, out var set))
{
set.Add(s);
}
else
var newSet = new List<string> {s};
words.Add(key, newSet);
}
}
}
}
words
is a Dictionary<string, List<string>>
, which we use to track the count of anagrams. The rest of the file loading is a fairly standard while
loop over the reader ReadLine
method, checking the dictionary to see if the anagram has already been found, and if so add the new word to the set, otherwise, add the anagram and create a new list to hold the word(s).
Once we have all the words loaded and matched into sets of anagrams, we can process them to work out which is the largest set with the longest word.
private static KeyValuePair<string, List<string>> ProcessAnagrams(Dictionary<string, List<string>> words)
{
var largestSet = 0;
var longestWord = 0;
var foundSet = new KeyValuePair<string, List<string>>();
foreach (var set in words)
{
if (set.Value.Count >= largestSet)
{
largestSet = set.Value.Count;
if (set.Key.Length > longestWord)
{
longestWord = set.Key.Length;
foundSet = set;
}
else
{
longestWord = 0;
}
}
}
return foundSet;
}
Here we simply bruteforce check all of the entries in the dictionary to find the answer. Itās not elegant, but it gets the job done. Running it on my Macbook Pro gives:
406627 anagrams processed from 466544 in 00:02:850
File read and key generation in 00:02:829
Anagrams searched in: 00:00:021
Found:
Key: AEINRST (7), Count: 15
aeinrst
antsier
asterin
eranist
nastier
ratines
resiant
restain
retains
retinas
retsina
stainer
starnie
stearin
Tersina
There are lots of blog comment systems, and this blog has used Disqus as the comment system for a long time. Iām not going to go into all the reasons to move away from Disqus, but page load times and wanting more control over your data and being able to respect your readers privacy figure highly.
Also, this blog is a technical blog focused on software development and associated topics, and this means that anyone who wants to comment on my blog is almost certain to be familar with Github and have an account, and also be as uncomfortable using Disqus as I have been.
I did investigate rolling my own code based on examples from other blogs, who have used some jekyll liquid templates and javascript to pull from the Github API and use it to post comments back to the repo hosting the blog. This has some attraction, but also has a big drawback, which is the authorisation situation to the Github API, as you donāt really want your client id and client secret exposed in the repo.
You can get around this by hosting an app in heroku to use as the postback url so that you can hide the client id and client secret, and there is also staticman, but none of these seemed as simple as just using utteranc.es
To configure utteranc.es, head over to the website and follow the instructions, and fill out the form to suit you. For the blog post to issue mapping, I chose āIssue title contains page titleā, and I also chose to have utteranc.es add a āCommentā label to the issue it creates in the blog repository. After you do that, youāll get a code snippet generated for you that looks somewhat like this:
<script src="https://utteranc.es/client.js"
repo="sgrassie/sgrassie.github.io"
issue-term="title"
label="Comment"
theme="github-light"
crossorigin="anonymous"
async>
</script>
Add this to a jekyll include, for example utterances.html
and then include it in your post.html
layout at the position you want the blog comments to appear. Most jekyll blog templates have Disqus support, so it will probably just be a simple case of finding where in the layout that Disqus is included, and replacing it.
If your existing comments are not important to you, then at this point you can stop and enjoy your new Github powered comment system. Personally for me, itās the principle of the thing, and the fact that the comments on my blog belong to me, and the author of the comment. So, we can do something about it.
Disqus allows you to export your comments, and once you do so, you will get your comments emailed to the email registered with your Disqus account. Iāve done a lot of work with XML in a previous role, and I think that the Disqus XML export looks⦠odd. The reason I say that is that each post on your blog appears to be mapped to a <thread>
element, which contains a bunch of expected metadata about the blog post. I would expect each individual comment to be a nested in a <comments>
element, but this is not the case. Instead, each individual comment has an entry as a <post>
element at the same level as the <thread>
, and they are mapped to each other using and attribute id. I donāt think that makes any sense, Iām sure there must be good reasons. I just canāt think what might be.
A comment then, looks like this:
<thread dsq:id="1467739952">
<id>218 http://temporalcohesion.co.uk/?p=218</id>
<forum>temporalcohesion</forum>
<category dsq:id="2467491" />
<link>http://temporalcohesion.co.uk/2010/10/25/lets-write-an-api-library-for-github/</link>
<title>Let&#8217;s write an API library for Github</title>
<message />
<createdAt>2010-10-25T12:00:24Z</createdAt>
<author>
<name>Stuart Grassie</name>
<isAnonymous>false</isAnonymous>
<username>stuartgrassie</username>
</author>
<isClosed>false</isClosed>
<isDeleted>false</isDeleted>
</thread>
An actual comment on this post looks like:
<post dsq:id="952258229">
<id>wp_id=25</id>
<message><![CDATA[<p>Great post Stu!</p>]]></message>
<createdAt>2010-10-25T22:47:44Z</createdAt>
<isDeleted>false</isDeleted>
<isSpam>false</isSpam>
<author>
<name>John Sheehan</name>
<isAnonymous>true</isAnonymous>
</author>
<thread dsq:id="1467739952" />
You can see the way that the post
element is mapped back to the containing thread
using the dsq:id
attribute.
The strange structure of the XML makes it less straightforward to parse the XML, as it means weāll have to do a little bit of work in matching up blog posts and the comments on them. Also very annoying is the fact that a thread
element doesnāt know if it actually has any associated post
comments.
We can acomplish this fairly easily with a little bit of F# and the FSharp.Data XmlProvider. Setting the provider up is straightforward, here Iām just using a direct reference to the assembly which Iād previously added via NuGet.
#r "../../.nuget/packages/fsharp.data/3.3.3/lib/netstandard2.0/FSharp.Data.dll
open FSharp.Data
type Disqus = XmlProvider<"/Users/stuart/Downloads/temporalcohesion-2020-07-13T20 27 09.014136-all.xml">
type Comment = { Author: string; Message: string; Created: System.DateTimeOffset; ParentThreadId: int64; }
type BlogPost = { Title: string; Url: string; Author: string; ThreadId: int64; Comments : Comment list }
let data = Disqus.Load("/Users/stuart/Downloads/temporalcohesion-2020-07-13T20 27 09.014136-all.xml")
If you are new to F# (and Iām still fairly new) this might look scary, but it really isnāt. After referencing the assembly in the script, we open the FSharp.Data namespace, and then initialise an XmlProvider by passing it the XML the file weāre going to parse.
That enables the XmlProvider to infer a lot of things about the XML in the file, and then the XmlProvider loads the actual data from the file. Two records are also defined to hold the details about the Threads/Posts that are going to imported, and how multiple comments map refer to a single blog post. These records are analagous to simple C# POCO classes with getters and setters.
With these types ready, we can define a couple of functions to convert the XML into them, and thus do a way with a lot of the extraneous noise from the XML, that we donāt really care about.
let toComments posts =
posts
|> Seq.filter (fun (post : Disqus.Post) -> not post.IsSpam || not post.IsDeleted)
|> Seq.map (fun (post : Disqus.Post) -> {Author = post.Author.Name; Message = post.Message; Created = post.CreatedAt; ParentThreadId = post.Thread.Id})
|> Seq.toArray
let toBlogPosts posts =
posts
|> Seq.filter (fun (thread : Disqus.Thread) -> not thread.IsDeleted)
|> Seq.map (fun (thread : Disqus.Thread) -> {Title = thread.Title; Url = thread.Link.Substring(0, thread.Link.Length - 1); Author = thread.Author.Name; ThreadId = thread.Id; Comments = [] })
These functions use currying, which as a longtime C# developer Iām still getting the hang of, and that will come in handy shortly. They map the Disqus
types generated by the XmlProvider into the custom types I defined, taking care to filter out comments we donāt want to import and not importing any blog posts which Disqus says have been deleted.
Seq.filter
in the toComments
function worked correctly, as I still had to go and manually delete a couple of comments that were marked as spam from the Github Issues
With those functions defined, we need a way of mapping the comments to the correct blog post.
let mapBlogToComments(post, comments) =
let commentsOnPost = comments
|> Array.filter (fun comment -> comment.ParentThreadId = post.ThreadId)
|> Array.toList
{post with Comments = commentsOnPost}
Here we take a single post, and all of the comments, and then use a nested function to grab the set of comments associated to that post, by way of the ThreadId
. With that written, we can use some more currying to create another function that will do a lot of hard work for us:
let addCommentsToTheirPosts comments = data.Threads |> toBlogPosts |> Seq.map (fun post -> mapBlogToComments(post, comments))
This function will take the threads, use the toBlogPosts
method to turn them into BlogPost
and then map each blog post to the correct comments using the method weāve just defined to do that. But where do the comments
come from? Well, it turns out this currying thing is really quite useful, as it enables all this magic looking |>
, or āpipingā to happen.
let toImport = data.Posts
|> toComments
|> addCommentsToTheirPosts
|> Seq.filter (fun x -> x.Comments.Length > 0)
Take all the posts data, turn them all into comments, and then pipe that to the addCommentsToTheirPosts
function, and then filter out blog posts which donāt have any comments, as importing those is pointless. All for around 24 lines of code. I know full well the C# it would take do all that, and whilst with C# 8 you could probably get close, I doubt youād equal 24 lines.
Just to be on the safe side, itās probably a good idea to look through each of the posts and comments that weāve now got to just to see if things are matching up correctly.
toImport |> Seq.iter (fun post -> printfn "%s - %s - comments: %d" post.Title post.Url post.Comments.Length)
Running that will give you an idea of what blog posts are going to be imported, and the number of comments. The first time I ran this, I found some of the blog posts in the Disqus XML import did not have the posts title set, so I was getting duplicated post titles. As there were only three instances of this error, I just manually corrected the XML and re-reran the script to check I had everything correct.
So far, so good. Now comes the fun part and something Iāve yet to do in F#, which is interop with a C# library. It turns out that itās not so hard, but that makes perfect sense when you understand that F# is a .net language, just like C#. A long time ago I started to write an API library for GitHub, but I gave it up in favour of Octokit.net.
We can easily reference Octokit and open the namespace as before:
#r "../../.nuget/packages/octokit/0.48.0/lib/netstandard2.0/Octokit.dll"
open Octokit
Then we just need to setup a few variables:
let repo = "sgrassie.github.io"
let githubApp = "foo"
let token = "<your-personal-access-token-here>"
let credentials = Credentials(token)
let header = ProductHeaderValue(githubApp)
let client = GitHubClient(header, Credentials = credentials)
These just get us a client to work with, and all I did was just register a new Personal Access Token on my account to use as the password. Notice how with F# you donāt need to new
anything, even though they are classes from a C# assembly. These can then be used in the following function, which Iām gonna prefix with this warning:
It does work though, so just⦠use at your own caution.
let exportToGithub posts =
for post in posts do
System.Threading.Thread.Sleep(2000)
let issuebody = sprintf "Comment thread for the post [%s](%s)" post.Title post.Url
printfn "%s" issuebody
let newIssue = NewIssue(post.Title, Body = issuebody)
let issue = client.Issue.Create("sgrassie", repo, newIssue) |> Async.AwaitTask |> Async.RunSynchronously
printfn "New issue created for %s" post.Title
for comment in post.Comments do
System.Threading.Thread.Sleep(2000)
let message = sprintf "Comment by **%s** on **%s** (imported from Disqus):\r\n\r\n%s" comment.Author (comment.Created.ToString("f")) comment.Message
let newComment = client.Issue.Comment.Create("sgrassie", repo, issue.Number, message) |> Async.AwaitTask |> Async.RunSynchronously
printfn " New comment created for %s" comment.Author
toImport |> exportToGithub
Iām sure that a more experienced F# person is going to look at that and be like āWTFā, but as I said, it does work. I left the printfn
log messages in, but essentially it loops over each post, waits a couple of seconds and then creates the new issue, and then loops over all of the comments for that post and adds then as comments to the issue. I put the Thread.Sleep
ās in the there just so I didnāt hammer the Github API, but honestly there was that few to import I doubt it would have trigged the rate limit, but I imagine a more popular blog with more comments on the posts woould.
Iāve been upgrading part of our build infrastructure to handle the ongoing upgrade to .net core, and as part of that, Iāve had to update the Cake build script to handle doing the restore in an offline environment, on the build server.
There is a great post on the Octopus blog about writing a Cake build script for .net core, I encourage you to check that out, Iām not going to repeat too much of that.
My specific requirement is that the `DotNetCoreRestoreā needs to succeed on an āofflineā build server, that is, a build server that has no access to the internet.
In order for this to succeed, you are going to need provide a way for NuGet to get the packages, usually this is done by maintaining an offline NuGet cache which you can point NuGet at, or even checking the packages into the repository. Iād always recommend going with the first option, although there are scenarios were the second option might be required.
However you do it, you need to tell NuGet where they are. The easiest thing to do is to use a NuGet.Config
local to the .sln
, but it is possible to code a location into the script.
Here is the restore task:
Task("Restore")
.IsDependentOn("Clean")
.Does(() =>
{
var settings = new DotNetCoreRestoreSettings();
if(BuildSystem.IsRunningOnTeamCity)
{
settings.PackagesDirectory = "./packages";
settings.IgnoreFailedSources = true
//optionally
//settings.Sources = new[] { "http://someinternalfeed/nuget" }
}
foreach(var project in projects)
{
DotNetCoreRestore(project.FullPath, settings);
}
});
This project uses a NuGet.config
to add the paths of internal package sources, and sets the location of the packages folder to be local to the .sln
- weāve found this cuts down on conflicts on developer machines.
In the previous post, I displayed my fledgling understanding of F# by writing a script which can parse the CSV set of results of the English Premier League to generate the league table. The script does this primarily by using a mutable BCL Dictionary
. F# is immutable by default, and whilst that is itself not immutable, you have to go out of your way to enable it. Iāll try to save repeating Scott Wlashin.
There are some improvements that can be made to the script. Iāll highlight them here and then link to the full script as a gist.
First a note on pattern matching. In the previous post I mentioned that I thought I could use pattern matching in a particular place, and obviously I can:
let fullTimeResult =
match row.FTR with
| "H" -> Home
| "A" -> Away
| _ -> Draw
Rather than if/then/else. Here, the āā is equal to the default
in a C# switch statement, if itās not a Home or Away (win), then it _must be a draw.
To start making things immutable, we can update the updateTeam
function from the previous post, and pass in a Map<string, LeagueRow>
:
let updateTeam (league : Map<string, LeagueRow>, team : string, points : int, forGoals : int, againstGoals: int, won : int, drawn, lost: int) =
if league.ContainsKey team then
let existing = league.[team]
let updated = {existing with Played = existing.Played + 1; Won = existing.Won + won; Drawn = existing.Drawn + drawn; Lost = existing.Lost + lost; For = existing.For + forGoals; Against = existing.Against + againstGoals; Points = existing.Points + points}
league.Add(team, updated)
else
let leagueRow = {Team = team; Played = 1; Won = won; Drawn = drawn; Lost = lost; GD = 0; For = forGoals; Against = againstGoals; Points = points}
league.Add(team, leagueRow)
The code is almost the same as the previous version, except that we no longer use the <-
operator to update the mutable dictionary. Whatās going on instead is that F# creates a new instance of the LeagueRow
, with updated values, and adds that to the Map
, by key, which has the side-effect of creating a new instance of the whole Map
, with the league row identified by the key replaced with the updated version.
The updateHomeWin
function becomes:
let updateHomeWin (league : Map<string, LeagueRow>, result : MatchResult) =
let league = updateTeam(league, result.HomeTeam, 3, result.HomeGoals, result.AwayGoals, 1, 0, 0)
let league = updateTeam(league, result.AwayTeam, 0, result.AwayGoals, result.HomeGoals, 0, 0, 1)
league
This again replaces the BCL Dictionary with the Map, and simply passes the league map through each updateTeam
call, and then returns the updated league object.
processMatchResult
is also updated to pass in a Map
, and calling the fold with it and a default map is straightforward:
|> Seq.fold processMatchResult (Map<string, LeagueRow> [])
This makes the script much more āthe way of thingsā in F#, which is to say itās using an immutable data structure.
During lockdown, Iāve made another effort at learning F#. This time I think Iāve had a bit more success. Processing data is something that we as developers do on a weekly or even daily basis, so it seems quite natural to practice that in F#. As a big football fan, Iāve decided to use the English Premier League results for season 2019/2020, as itās a dataset I implicitly understand.
The EPL results set is available in CSV format from football-data.co.uk, and rather than having to parse it all by hand or hitting up CsvHelper and still having to write some C# code to actually use it, in F# we can use a Type Provider, specifically the CsvProvider from FSharp.Data.
FSharp.Data is easily added via NuGet, and using an .fsx
script, we can easily reference the assembly and open the namespace:
#r "../../.nuget/packages/fsharp.data/3.3.3/lib/netstandard2.0/FSharp.Data.dll"
open FSharp.Data
open System.Collections.Generic
I didnāt have any luck in the script with referencing a more local copy of the assembly, such as one in the /bin
folder, due to it complaining about not being able to find the FSharp.Data.DesignTime.dll
, but going directly to the assembly in the NuGet packages folder seems to work just fine. It is also worth noting that Iām writing this on a Mac (in VS Code), so your path syntax might vary. Also note that we also open the BCL System.Collections.Generic
namespace. Weāll need that later.
Next, comes the part that blows my mind. Here is how we generate a type which knows how to load and parse a CSV file of a given structure:
type Results = CsvProvider<"../../Downloads/epl1920.csv">
Thatās it. Itās pretty amazing. The Results
type is now also type safe, and itās had a guess at infering what the types are for each column of the data. We could probably do something similar to this in C# using CsvHelper
and either Castle.DynamicProxy
or some magic with the new Roslyn compiler, but I think it would take quite a bit of code to create something that came close to what this can do.
Skipping over some important stuff that weāll get to in just short a while, we can now easily load the full results set:
Results.Load("../../Downloads/epl1920.csv")
This is fairly straightforward, and does exactly what it looks like. The data loaded from the file is available in a .Rows
property, that weāll use shortly.
All good so far, but now things get a little more complicated. Now we need to think somewhat about the data, and if you look in the file⦠itās got a LOT of information. Mostly related to betting information for the match, but there is also quite a lot of information about the match itself. For the purposes of calculating the league, most of thie information in the file is redundant. In order to just get the information we need, we can define a Record to hold to that information. A Record in F# is somewhat analagous to a C# POCO class, but with automatic type safety and full equality comparisons out of the box.
type FullTimeResult = | Home | Away | Draw
type MatchResult = {HomeTeam : string; AwayTeam: string; HomeGoals: int; AwayGoals: int; Result: FullTimeResult}
The FullTimeResult
type is just like a C# enum, and is easier to read than the āAā, āHā or āDā we get from the CSV file for the FTR (Full Time Result) column. I think it also looks nicer to read when it comes to the pattern matching, but weāll get to that. With those types defined, we can get to the real meat of this and actually parse the data:
let league = Results.Load("../../Downloads/epl1920.csv")
.Rows
|> Seq.map toMatchResult
|> Seq.fold processMatchResult (Dictionary<string, LeagueRow>())
|> Seq.sortByDescending (fun (KeyValue(_, v)) -> v.Points)
Here, we load the file as we discussed earlier, but now, we forward pipe the data returned from the .Rows
property to Seq.map
through the toMatchResult
function, which takes a Row
and extracts the data weāre interested in and returns a new MatchResult
. In C# this is the same as doing .Rows.Select(new MatchResult {...})
. Then, the resulting sequence of MatchResult
s is piped forward through the processMatchResult
function, using the scary sounding Seq.fold
, and it also passed a new instance of a BCL Dictionary
, with a string
key and a LeagueRow
type as the value. Iāve not yet mentioned the LeagueRow
type⦠itās not super important to proceedings, it just a type which holds all the data you would expect to see in a football league table. For reference itās included below in the full script.
Amazingly, those five lines load the file, process all the data, and provide an object which contains a fairly accurate version of the English Premier League table. Obviously things are a little more involved than that.
As youāll recall, the there is a lot of data in the CSV file that is irrelevant when it comes to generating the league table. We can map all the data we need into the MatchResult
type, which we do by forward piping the data through Seq.map
and the toMatchResult
function:
let toMatchResult (row: Results.Row) =
let fullTimeResult =
if row.FTR = "H" then FullTimeResult.Home
elif row.FTR = "A" then FullTimeResult.Away
else FullTimeResult.Draw
{
HomeTeam = row.HomeTeam
AwayTeam = row.AwayTeam
HomeGoals = row.FTHG
AwayGoals = row.FTAG
Result = fullTimeResult
}
This is mostly just a simple mapping from the results row into the new MatchResult
type. Youāll notice we donāt need to explicitly ānewā anything up, donāt forget, weāre in a functional world now, so the MatchResult
is returned as a side affect of what weāre doing. We also define a nested method which processes the full time result using a simple if/else construct. I think I could also have used pattern matching, but itās simple enough that Iām not going to worry about it.
Next, comes the scarying sounding fold. The method looks like this:
let processMatchResult (league : Dictionary<string, LeagueRow>) result =
match result.Result with
| Home -> updateHomeWin(league, result)
| Away -> updateAwayWin(league, result)
| Draw -> updateDraw(league, result)
league
What happens is that we tell Seq.fold
to use this method to do the folding, and we give it an initial state of a new and empty Dictionary<string, LeagueRow>()
. Seq.fold
carries the state over to each subseqent āfoldā over the sequence of MatchResults
it was piped. Youāll note that the final thing returned as a side effect of the method is the same dictionary which was passed in. This essentially forms the core of the algorithm to produce the league. The pattern matching of match <thing> with
is equivalent to a C# switch statement on steroids. I am barely scratching the surface of what can be done with pattern matching in F#.
The patten patch decides what kind of result we are dealing with, and delegates further processing to the relevant method. Here is the definition for updateHomeWin
. The other two methods are exactly the same, except they distribute the points/goals/wins/losses/draws accordingly, so I wonāt go into those in detail.
let updateHomeWin (league : Dictionary<string, LeagueRow>, result : MatchResult) =
updateTeam(league, result.HomeTeam, 3, result.HomeGoals, result.AwayGoals, 1, 0, 0)
updateTeam(league, result.AwayTeam, 0, result.AwayGoals, result.HomeGoals, 0, 0, 1)
Each MatchResult
consists of two teams, and we have to update each entry in the league for both of these teams, with the correct number of points, goals for, goals against, win, draw and loss. The real part of this is in the updateTeam
function:
let updateTeam (league : Dictionary<string, LeagueRow>, team : string, points : int, forGoals : int, againstGoals: int, won : int, drawn, lost: int) =
if league.ContainsKey team then
let existing = league.[team]
let updated = {existing with Played = existing.Played + 1; Won = existing.Won + won; Drawn = existing.Drawn + drawn; Lost = existing.Lost + lost; For = existing.For + forGoals; Against = existing.Against + againstGoals; Points = existing.Points + points}
league.[team] <- updated
else
let leagueRow = {Team = team; Played = 1; Won = won; Drawn = drawn; Lost = lost; GD = 0; For = forGoals; Against = againstGoals; Points = points}
league.Add(team, leagueRow)
This is just a simple dictionary update where we check if a team already has an entry, and if so, update it, otherwise we create it. Things of note here are that whilst F# is mostly immutable, types from System.Collections.Generic
are mutable, which is how this whole thing works. Iām sure that someone much better at F# can come along and tell me how to do this with immutable F# collections. Also of note is the collection access of league.[team]
, which is different than in C#. We also update the value in the dictionary by using <-
.
After that, we can define a simple method to print out a row from the league for us, and then iterate through the entries in the dictionary, to get a league table:
let print league =
printfn "Team: %s | Played: %d | Won: %d | Lost: %d | Drawn: %d | For: %d | Against: %d | GD: %d | Points: %d" league.Team league.Played league.Won league.Lost league.Drawn league.For league.Against (league.For - league.Against) league.Points
league
|> Seq.iter (fun (KeyValue(_, v)) -> print v)
The KeyValue
is an active pattern, which matches values of KeyValuePair
objects from the BCL Dictionary, and this produces (with data correct as at the publication of this post):
Team: Liverpool | Played: 33 | Won: 29 | Lost: 2 | Drawn: 2 | For: 72 | Against: 25 | GD: 47 | Points: 89
Team: Man City | Played: 33 | Won: 21 | Lost: 9 | Drawn: 3 | For: 81 | Against: 34 | GD: 47 | Points: 66
Team: Leicester | Played: 33 | Won: 17 | Lost: 9 | Drawn: 7 | For: 63 | Against: 31 | GD: 32 | Points: 58
Team: Chelsea | Played: 33 | Won: 17 | Lost: 10 | Drawn: 6 | For: 60 | Against: 44 | GD: 16 | Points: 57
Team: Man United | Played: 33 | Won: 15 | Lost: 8 | Drawn: 10 | For: 56 | Against: 33 | GD: 23 | Points: 55
For completeness here is a gist of the full script:
Suppose you’re on a rocket ship, and you’re given the choice of three buttons: one button starts up your advanced MHE thrusters, but the other buttons explode the ship. You choose some button (say, number 1), and the rocket ship itself (which is a sentient AI), who knows what the buttons are connected to, chooses another button (say, number 3). It then says to you, “Do you want to choose button number 2?”
Is it to your advantage to switch your choice? Also, who designed this control panel?
I second this. I’ve been taping my mouth closed at night for the past year…
In every introduction to a potential client, partner, or other associate, the first thing I do is give a brief overview of my history. I know this is common in just about any business or social interaction, but its especially important in my line of work, since communicating my curriculum vitae is so critical to demonstrating my technical competence. And in truth, a technical lead can be as charming as you’d like but unless they are extremely technically competent, nothing else matters. But its hard to compress a lot of data into a short introduction!
I’ve worked on projects for some of the biggest companies in the world — not just Fortune 500 companies, but Fortune 50 companies! — as well as launched dozens of startups, some that didn’t work, some that did, and some that were extremely successful. Those larger companies include industry giants like Disney (three divisions — Disney Parks, Disney Channel, and Disney film!), Dreamworks, Warner Bros., Accenture, CBC, Sega, Sonos, and many more, as well as successful, venture backed startups like Graphite Comics, for which I currently serve as CTO.
I’ve also been written about, and my projects (both professional and personal) have been written about, by publications like the New York Times (which covered the launch of Graphite Comics, and the AI recommendation system I built for it, on the front page of the Business section), The Guardian (twice!), Fortune, TechCrunch, USA Today, AdWeek and many other internationally known outlets, in addition to multiple smaller but no less influential blogs and online journals like PocketGamer, TouchArcade, VentureBeat, and many, many more.
Just typing up that last paragraph and linking to all of that old press was a proud moment. In addition to mainstream press, I’ve also published papers on game related AI, including writeup of my graduate thesis on Hebbian learning in artificial neural networks — a topic I want to get into in more depth later.
That being said, I get asked a lot to dive deeper into my background — not only what I’ve worked on in the past, but why and how, which, to me, are much more interesting questions!
(Note: I will be adding to this as time permits until I’m finished!)
When I first became interested in computer science and programming specifically, there really wasn’t much of an industry per se — at least not anything that looks anything like the industry today. What did exist then was a collection of business-centric hardware and software providers for the most part, building tools to help businesses do the kinds of things businesses did in the 80s — spreadsheets, simple printing software, things like that.
If I try to remember exactly why I became fascinated with computers back then, I genuinely couldn’t tell you! There was very little for me to sink my teeth into — and keep in mind I was a very young kid at this time — certainly not interested in spreadsheets and early databases!
I guess I can liken it to those early video game experiences. Why was it fun to make the white dot chase the grey dot? Why were any of those Atari 2600 games I played as a kid fun in the least? I also can’t answer this question, except to say that in both cases, I think I, and all the other kids intrigued by tech back then, just sensed that something cool was in there. We knew that someday the ultra simple games we played like Pong and Pac Man would mature into nearly photo-realistic, open world masterpieces like GTA V. Maybe we sensed those big, boxy, monochrome machines that just displayed green text would someday fit in our hand, and allow us to do amazing things like fly a drone, video chat with someone around the world, or find our way home.
Whatever the case may be, I found myself drawn to technology in a major way around the age of 10. My neighbor friend’s dad worked for IBM, and he had an early IBM PC that didn’t do much, and I had a few friends from early on who had early Apple computers. But my very first computer was a Commodore 128.
The Commodore 128 was an upgrade from the immensely popular Commodore 64. I believe the Commodore 64 is still the biggest selling single computer of all time. At any rate it was a very popular machine, primarily because you could run games on it — and the games it ran were pretty amazing, especially for the time. Even better, you could plug an Atari 2600 controller right into it.
The C128 came with BASIC, which was my first programming language. I’m actually surprised BASIC, or something like it, isn’t more prevalent these days. BASIC is, after all, extremely basic, and it was a great way for 10 year old me to start learning the essentials of programming. The C128 came with a few big thick manuals, and one of them was a complete guide to programming the machine in BASIC.
This is one of the most interesting and stark differences between the early days of PC use and today. Computers shipped with compilers and manuals explaining how to use them. Imagine if every iMac you bought came with a big book on how to code in Swift and XCode not only preinstalled, but tightly integrated into the operating system! That’s what things were like then — if you shelled out the money for one of these things, it was more likely than not that you planned to build software for it.
But while BASIC was fun, it didn’t take me very far. You really couldn’t do much with BASIC, so I found myself focusing on generating sounds with it. One of the demo programs in the manual (printed out, so you had to copy it line by line off of paper to get it into your computer!) was a program to play a piece of music by generating tones with the C128s pretty amazing audio chip. But other than that — well, without a lot more skill than I had, there wasn’t much farther for me to go.
So I switched my focus to networking. That little C128 had a slot you could plug a modem into, and you could use it to dial into a few online services. Some of these were early Internet-like things that let you play games or talk with people, and as I reflect back on them, they were really pretty amazing pieces of software for the time.
So when I finally upgraded to an IBM compatible PC, the first thing I did is install a modem, and then start to tinker around with Wildcat BBS. I quickly met up with a group of local (Orange County, California) tinkerers who were using Wildcat to connect with other enthusiasts, and eventually became one of the first members of my school’s computer club — where we mostly just messed around with Wildcat. There still wasn’t much out there that was fascinating on the consumer level, at least not, to me, moreso than BBS software. The idea that I could link up with other people through my computer was amazing.
But my time with Wildcat didn’t involve any coding, and programming was what I really wanted to learn.
Eventually, I went to high school, just in time for the school to offer an AP Computer Science course, which I took my junior year. At that time, the course was taught in Pascal, which was a much more complex language than BASIC and I quickly started to imagine the possibilities with this new language. Pascal lead me to C, which led me to C++, and then to Java in its early days. Since my high school only offered the one CS class, I took some college level classes in high school at Fullerton College and California State University, Fullerton in Unix, C, C++ and a data structures course taught in Pascal. I was actually starting to become a polyglot at an early age, which I’m very grateful for, as I know a few very talented developers who struggle learning new languages and platforms. Being thrown into so many so young (mostly because the industry was all over the place back then!) really helped me later on, I think.
I started school at the University of California, Santa Barbara as a double major — Mathematics and Computer Science. This was so early on that very few students even had an email account, as everything was done through dial up into a Unix system using text-based Unix utilities like mail, finger, talk… You could look things up in a really ridiculous way using gopher, archie and eventually on the web using lynx. But then, I was lucky to get one of a very few SLIP accounts, and eventually a PPP account. Don’t even bother looking those up, it was a short lived thing but it was how you could jump on the graphical internet over dialup. Using those technologies, I was the first person I knew in the entire world who could look things up on the web, in an actual browser, with images and everything. And imagine how life changing that was.
I feel like people in their 20s just really can’t imagine how Earth shattering this all was back then.
Anyway, I ended up only taking the Mathematics degree, as I planned on graduate studies in Computer Science and wanted to get to that as soon as possible rather than to an extra year as an undergrad. Specifically, I had planned to chase a PhD in Computer Science and then figure out where to go from there. But then something strange happened…
I went to film school.
I’m not sure when it happened, but it occurred to me that one of the things I loved the most about technology was the creative side of it. I loved games, and music, and film, and all of these wonderful things that you could do with computers suddenly in addition to writing interesting software on them. And I wanted to hone my creative skills. So I started an MFA program in film production at Chapman University.
After school I moved to Los Angeles and ventured briefly into the creative arts — first as a struggling filmmaker, then as a punch up writer (I would be hired to take scripts that didn’t quite work and add jokes or interesting scenes to them). Eventually I got into some support work with a few local film festivals. But I really didn’t enjoy the film industry at all so I started dabbling in something I did enjoy: music.
Now this is going to sound like I was all over the place, and I was. But I started writing and producing electronic music, and pretty quickly started getting hired to mix and produce CDs, remix songs for some well known artists, and eventually even landed a record deal. But the whole time I was more interested in the idea of pushing creative boundaries than anything, and since I was working in the electronic music arena, the way you did that was with software.
So suddenly I found myself back in the software world — this time writing music software. Specifically, I was building virtual effects and instruments using a technology called VST and VSTi. This technology allowed you (and still does to this day) to build instruments and audio processing plugins for any DAW that supports the VST format.
I quickly found myself spending more time writing software than music, and thus I was thrust back into the world of technology.
Around this time I took a job as a network engineer at a company in Orange County, and later in downtown Los Angeles. The life of a network tech is dull, but I did it because it gave me access to two things: tons of computer hardware, and immensely huge Internet bandwidth — the latter of which was tremendously expensive and tremendously hard to get back then.
I used those resources to do two things: first, to run a series of Internet radio stations (yes, “Radio on Internet”) called the Glowdot network, and to build a photo sharing site called Glowfoto.
Glowfoto eventually took over my life, and I developed it into a full-scale social network. I was lucky to ride the MySpace wave early on, and Glowfoto became an early companion site to MySpace. Glowfoto allowed MySpace users to upload more than 10 photos to their profile, back when MySpace had a 10 photo limit (if you can believe that).
Eventually MySpace went away, and about 5 years later I finally retired Glowfoto as well. But during that time I started getting many, many requests for me to build similar services for other companies. And thus my career as a contract software developer began.
(more coming soon!)
I usually like to keep my posts more positive-focused — here’s what you should do vs. here’s what you shouldn’t do. But this week alone I had three potential clients relay to me a very common experience:
I thought I finally found a good developer, but then they suddenly just disappeared!
I’ll admit its hard to explain to entrepreneurs sometimes why that developer ghosted them — usually the explanation feels personal, or accusatory. But in my experience its just a matter of communication! Often its something that comes up in the initial conversations that seems innocuous to the person looking for a developer, but to that developer, its a huge red flag.
In this post, I’ll try to give a few of the reasons that developer may have disappeared into thin air.
I think possibly the best way to explain what’s going on here is to break down a few common statements, and look at what the entrepreneur probably meant to express, and how the candidate developer interpreted it.
However, its important to note that sometimes what was said was exactly what was intended, and what was heard was exactly what was intended! In those cases, I think typically the problem is that the entrepreneur just doesn’t quite understand the market they are entering. You’ll see what I mean when we get to one of those cases.
I also think its important to state a few very important facts right out of the gate.
Most important among these is that contract software developers are still hugely in demand, and hugely in short supply. This may not seem to be evident when you put out an RFP and get snowed with replies. Its important to keep in mind that the vast, vast majority of those replies are from incompetent or just junior developers, non-technical middleman project managers who are going to outsource your project, or direct offshore dev shops staffed by ludicrously unqualified programmers.
When you strip away the worthless proposals and just look at the qualified, experienced developers who are actually capable of building your app, you quickly realize there is a tiny handful of qualified, experienced freelancers out there compared to the massive number of RFPs put up daily.
It’s important, then, to understand that when you do finally zero in on that great developer you should convey an understanding and respect for his or her talent, time and attention. Far too often, good developers get spoken down to by potential clients as if they are in that same pool of dreck that I mentioned above. And in fairness, from the client’s perspective, that might as well be true for the initial portion of the conversation. But it’s still not an encouraging attitude to take with someone you are about to entrust with the technology powering your business!
Another reason for the apparent discrepancy is that to date there still hasn’t been anyone to come along and crack the problem of pairing quality projects with quality developers. Some companies like Upwork have taken great strides in improving the way they match candidates to jobs, but it is still pretty far from perfect, and I’ve found that far too many decent jobs are getting 50+ responses from horrifying developers. That’s not to mention the insane number of very low quality jobs from completely unfunded clients that get posted daily.
Why is this important? Because some potential clients are initially confused why a developer would aggressively pursue them, only to vanish into thin air. After all, if you needed the work bad enough to contact me, the thinking may go, then you must need me more than I need you.
However, that’s seldom true. Instead, because that matching has not been solved, developers often need to cast a wide net to find quality clients, just as clients need to cast a wide net to find a quality developer. So in the end, both parties have been sifting through enormous piles of potential candidates and clients to find the one good one in the bunch.
And for that reason, incidentally, I plan to do a follow up post to this one about why that client ghosted the developer! Believe me, this situation goes both ways.
So anyway, there is still a lot of work to do. But the most important takeaway here, and something I must absolutely stress in the extreme is: there are very few good developers out there, and very many jobs. Once a developer connects with a potential client, it becomes an immediate game of filtering the good from the bad, and trying to determine which employers will be a pleasure to work with, and which will be a nightmare.
Remember: its not just you interviewing a developer. They are also interviewing you!
Ok let’s run through a few of the things you might say that will scare that developer away.
What you probably meant to say: I’m not trying to build a nuclear submarine here.
Oof. Gotta get the most common and possibly the worst one out of the way first.
Here’s the deal. If you aren’t a developer, then you most likely don’t really know how long anything takes. That’s one of your developer’s jobs, in fact — to help you estimate time and cost.
But the big problem with a client who doesn’t know how long things take but thinks they do is that those clients will never be happy. Everything will feel like it is taking too long and costing too much money if they think it’s “easy” and “should only take a couple hours” but is in fact hard and takes a long time. And the developer can expect a constant stream of frustrated emails to that effect.
Secondly: almost nothing takes “a couple hours”. Maybe a quick little bug fix — assuming you know where and what the bug is. But otherwise, no.
So this is a bad assumption to make right out of the gate. And although you probably mean this to say “hey, I’m not trying to make your life tough with this!” what the developer hears is “I’m going to make your life a living hell”.
Seriously.
What you probably meant to say: I only need a developer for a few hours to work on something small.
Here is one of those examples of the client saying exactly what they mean. Absolutely nothing wrong with that! And yes, this looks a lot like the last one. It is! Just without the accidental insults.
But the fact is, all you need is a couple hours of that developer’s time! However there are a few considerations to keep in mind
First, that gig that takes just a few hours also requires a few hours to set up and tear down. In development, there is a time cost to jumping into something new that is often hard to explain to a client.
It can take an hour or two just to discuss the requirements of the job. It can take a couple hours to download the current project, open it up, poke around, get everything set up in the dev environment, etc. Then, finally, there is a cost to getting out of the project — delivery, invoicing, debrief meeting, etc.
Are you ok with being billed for those hours? A 2 hour quick in-and-out project could actually take 4-5 hours given the above. Some developers I know who would otherwise take on small quick gigs like this actually end up passing because they find it really hard to explain those extra hours to clients, and it ends up not being worth it.
Second, most freelancers are looking for something long term, not very short term. They’re looking for the weeks-long or months-long project that will sustain them for a while. It would be incredibly brutal to put in all the legwork involved in pairing up with a client a hundred times a month to gather 80 or so 2-hour gigs a month just to earn a sustainable living. In fact, its not even humanly possible — this is a subject for another post, but you might be very surprised to learn the actual cost in time and money involved in securing a client. Suffice to say it is much too high to allow for too many extremely short term clients in the “couple hours of work” range.
And so, unfortunately, these projects often get dropped — even though they very well may turn into more work in the future!
I don’t have any good suggestions for this one. Unfortunately this just is what is is. I can say that eventually you will find someone willing to help you out for that short term. It just might take longer than you expect!
What you probably meant to say: Help!
Ok this one is interesting. From your perspective, “taking over” a project probably seems like a pretty normal thing. However, from a developer’s perspective, a whole lot of alarm signals go up right away.
Here are the first questions that immediately pop into my head:
Did they quit? Why? Did they get frustrated? Did the project go to hell? Did the client start making unreasonable demands or developer unrealistic expectations? Did the client start demanding a lot of free work?
It is important for the developer to have an understanding of what working with a new client might be like, and you are providing a piece of potentially valuable information here: someone was working for you, and now they aren’t. Something, surely, went wrong.
In order to mitigate against this, be upfront about what happened! Get ahead of this question, and let the prospective developer know what happened to the last one.
No one wants to walk into a project that already has problems. In the best case, software doesn’t have bugs. It isn’t riddled with problems and unknown issues that need to be tracked down.
Now, in the real world, of course there are bugs to be squashed, almost always. But in my experience, the kind of project that starts with “I want to hire you to fix some bugs” is utterly rife with bugs and faults. And walking into that scenario is simply not appealing to any developer.
Mitigate this question by being transparent about the situation. Is the app functional, but there are just some small usability issues? Or is the app crashing constantly and losing data like crazy?
What sort of headspace is this client in?
Are you completely frustrated? Are you now convinced all freelance developers are as incompetent as the offshore developers you just hired for $20 an hour?
If you just had your money lit on fire by a worthless team of offshore developers, and you are now looking for a local developer to fix the situation, its important that you understand the vast difference between the two.
A good local developer is college educated, possibly at the graduate level, and has real experience, has shipped highly rated apps and earns a real living doing this.
The offshore developers you just worked with barely have a highschool education in some instances, and took a 3 week coding crash course before being shoved in a room and told to write code for projects they barely understand. I’m not kidding about this.
So make sure you don’t treat the developer who is potentially going to save your product the same way you want to treat the people who just burned it to the ground.
I have had potential clients straight up tell me they don’t value developers at all and see us all as incompetent wastes of space. And while I really, truly do understand their frustration, having lost it all by going with the hilariously bad cheap option — do you think I had any interest in taking on that job?
Yikes.
No thanks! I’m here to help, not to be insulted. I have a MS in Computer Science and left the Ph.D. program at UCLA to do the work I do now. I’ve been doing this for over 20 years. I am not on the same planet as the $20/hour developer you just hired with 3 weeks of bootcamp training.
Please keep this in mind when you are interviewing prospective onshore developers!
What you probably meant to say: this is going to be tough, so I’m willing to pay top dollar.
This one comes up a lot. There are certain types of software development that are just hard work out of the gate. Anything that requires compliance with government regulations, opens up clients or companies to liability of any kind, apps dealing with copyrighted material, etc.
Some of the absolutely best ideas I’ve heard over the years involve the healthcare industry or the public education sector. However, in my experience these projects are brutal and come with huge obstacles right out of the gate!
Sometimes the client just doesn’t know what they are getting themselves into. That happens. But more often than not, they do.
The bigger issue here is that there are other jobs out there that simply don’t come with the headaches of compliance and liability. In order to make these jobs appealing, they just have to come at a higher price, unfortunately.
Sadly, I have known some good clients in the past that did not bring their budgets up to the level required for the quality work required for a project like this, and eventually went offshort for low-quality, low-cost work. In literally every single case, they either ended up with no app at all, or an app that utterly and completely failed to comply with the regulations.
Let me just stress again: these apps are tough. Not only are they stressful for the developer, they require a lot of experience, and that experience, unfortunately, is expensive.
What you probably meant to say: my industry is important to me, and I want it to be important to you too.
This one is more humorous to me than anything. But I know a few developers that use this to screen out potential clients.
Here’s the thing: you may be an expert in the sneaker flipping industry, but I’m an expert at software development. You are massively unlikely to find a good software developer who has focused on your particular industry. It just doesn’t work like that.
We jump from project to project by the very nature of what we do. Sometimes we make social apps, sometimes B2B apps, sometimes fintech, sometimes e-commerce. I literally don’t know a single freelance software developer who has built a career focusing on only one industry.
Oh and that sneaker example? That’s one I just saw recently. They wanted a developer who was a sneaker fanatic. And you know what? They may find one, because at least that exists and there may be some overlap. But generally speaking, you really shouldn’t limit the pool of candidates you are interviewing by insisting that they have particular expertise in your field.
And furthermore, it almost never actually matters! Unless you are building a complex AI driven system, in which case experience with AI would be mandatory, or if you are building an image processing app, in which case image manipulation experience would be important.
But the key is: unless you are specifically seeking someone with experience in the particular technology you are developing, then the experience really is unlikely to matter much.
If you are building an e-commerce platform for buying and selling medical equipment, then experience building storefronts, taking payments, managing invoices, etc would be a huge plus. Experience in the medical industry would not.
What you probably meant to say: last time I posted this I got a bunch of weird responses from Indian developers and wasted a whole bunch of time.
I am very sympathetic about this one. However, it is generally not a good idea to treat potential hires like peons just because you had to deal with a bunch of jokers the last time you tried this.
I completely get it though — you will get flooded with responses from low quality developers. However, it is a very, very bad idea to let the one good developer think you have been soured and now consider all developers to be annoying garbage.
I think this one sort of speaks for itself. If I’m deciding whether or not I want to take this project, the last thing I want to risk is working with someone who has a chip on their shoulder!
What you probably meant to say: is this feasable?
The short take on this is simple: the apps you are trying to replicate didn’t start out as complex as you see them now. And unless you want to spend the time and money they did to get there, you should probably rethink this plan.
Instagram, for example, didn’t have video and stories and IGTV, or even gallery posts. It was a dead simple feed of square images bundled with a couple very simple filters.
I replicated the core functionality for two apps i built for Warner Bros. to promote two films several years ago (Tim Burton’s Dark Shadows, and Christopher Nolan’s Dark Knight Rises). And I built the first of those apps in 3 weeks on iOS — and this was before Apple released the CoreImage library.
How? Well because Instagram was pretty simple back then! It took me a couple hours to reverse engineer how the filters were assembled, build a little framework for configuring these filters and applying them to an image, and voila!
However, if you asked me to replicate Instagram now, I’d probably run for the hills. Instagram is complex now! But they got there after many years, many millions of dollars, and an acquisition by Facebook.
See its not just the time requirement. Some of these features are just not feasible when you are still trying to break your development into bite-sized fixed price chunks and having a freelancer build them piecemeal. At a certain point, you need to actually start to structure your company like a company — and that means raising capital to build these functions, hiring full-time engineers instead of bouncing from freelancer to freelancer, and organizing and coalescing your team — engineers, project managers, designers, et al — under one roof, so that they can effectively work as a unit to build the project according to the bigger vision and roadmap you have.
This is a much longer post as well, but to keep it brief: it’s just not realistic to work the way you did at the beginning all the way through this complexity, as your requirements become more and more complex. The first version of Instagram can be built by a freelancer. The modern day version of Instagram requires Facebook.
Now, why is this a problem for a developer? Because by simply asking for this feature in this timeline, they can tell right away that you have unrealistic expectations — not only in terms of the time required to develop these features, but the company structure and funding required to do so. There is a good chance that you don’t have a real solid grasp of what exactly you are asking for, and just how big a task it is, and that can make your project tough to work on!
A lot of developers have come to learn that clients with unrealistic expectations often can’t be talked out of those expectations. And, so, they typically just move on.
What you probably meant to say: I have no money
It’s ok to take a shot at bringing your vision to life, even if you don’t have enough money to do it exactly the way you’d like. It really is! I am in this business because I love working with visionary people with big, outrageous dreams. The tech industry is a thrilling, vibrant and lucrative place to work because those people exist.
What’s a little far out though is the client who is not doing the math at all.
I had a meeting this morning that went exactly like this. The client needed an iOS app, Android app, web app, backend REST API and custom web-based admin panel. The client understood it would take anywhere from 3-6 months to do this in the best case, and even seemed to understand I would need to bring in help to do it in that timeline. Then told me with a straight face that the budget was $3000 firm.
So I did the math. Two developers working around the clock to get this done in an extremely optimistic 3 months works out to about $3 an hour each. Considering the 3 months was, as I said, very optimistic in this case, I would project closer to $1-2 an hour.
And they were specifically looking for a developer based in Southern California. Well, I’m in Southern California, and I can tell you if I made $3000 a week I’d be living a dumpster next week. I don’t even see how I could make it past week three if I was earning $1000 or less a month. Its expensive here. And even if it wasn’t, that is a ludicrous amount of money for anyone, anywhere.
But not only that, you are hiring highly skilled, highly trained, in demand people to do this work. Or, at least you should be if you care at all about what you’re building! But you are expecting them to leap at a job that pays about 5% of the local minimum wage.
I don’t even know what else to say.
What you probably meant to say: I have no money
Let’s finish with the big deal breaker. This, too is a post all on its own, but its a big one and it comes up all the time.
There are a few other common and related statements I can throw in with this one:
Really quickly, let me break down a few things.
First, this is our job. We aren’t a charity giving up our time to help you strike it big. We have bills to pay and loved ones to take care of, and reality just doesn’t permit us to take on projects pro-bono. It just doesn’t, and I don’t know anyone who knows what they are doing who ever jumps on board on an equity basis at the early stage.
Again, there are a lot of reasons for this, and hopefully I’ll get to post about them one day. But for now, the simple version is: you just aren’t there yet.
The time to ask a developer to come on for equity is after you have a product, and you have some traction in the market. NOT when all you have is an idea, or some awful prototype you had build overseas that barely works.
While I’m absolutely sure that you are super excited about your idea, you’re asking a developer to be just as excited as you are, when in reality all you have is an idea. And to someone who listens to ideas all day long, an idea alone just isn’t that exciting. What is exciting is when that idea starts to work. When users start to react positively to the released product. When the non-technical team starts to put together those critical deals and build those incredibly important relationships that take that humble idea and turn it into a viable company. That’s when you should ask someone to come on for equity, not before!
And typically, equity offers come along with pay, not in lieu of pay. In fact, its your primary job as an entrepreneur at the earliest stages to raise money for your company! I know its hard — believe me, I’ve been through it many times. But it’s a crucial part of launching a company, and it’s critical to building a competent team that will be able to build the technical foundation for your company.
But until I write that big post explaining this in more depth, let’s just keep it short and simple: you get what you pay for.
One of the advantages (if you can call it that) of being in this industry as long as I have is that I’ve been through multiple economic disasters, technological paradigm shifts, and — it’s not all bad! — economic and technological boom periods.
So I figured I might as well throw out a few predictions as to how I feel technology, and more specifically the app world, is going to change after this is over. What kinds of apps will consumers want, what kinds of apps will entrepreneurs build?
Keep in mind this is just my opinion based on my own experience, and should not be taken as anything but that. That being said, lets start with the big question: what sorts of apps are going to be big after the coronovirus scare of 2020?
One of the initial download spikes I saw after people here in California were asked to stay home was Zoom. Not just Zoom though — Skype, Discord, and other communication apps suddenly became much more useful as we all retreated home.
What’s interesting to me in times like this is how people start bending existing technologies to their needs. For example, you may have an app that was used primarily for virtual face-to-face conferencing for businesses now being used to host virtual get togethers for people under lockdown.
That’s an interesting opportunity for entrepreneurs to start to see gaps in the technological landcape. The market will tell you really quickly that there is a need for a specific app tailored for that particular use case. Its possible that platforms like Zoom will step up and fill the gap themselves. But sometimes those apps can’t or simply won’t, and that’s a huge opportunity to get in and fill an existing need.
One of the biggest struggles I see with many app startups is that they created something very cool, very interesting, but not very in demand. That means when they launch, they not only need to find users and let them know they exist, they also have to explain to them why they should care. It’s always an easier road when the market is already waiting with baited breath for your product.
I know the social networking landscape is already pretty crowded, but I predict we’ll see a few more innovative apps in this space in the next year or two.
I have been watching a few giant gaps of my own for the last few years (and hopefully one day I’ll have time to set about filling them!) but more are sure to pop up as the battle against Coronavirus goes on.
Many of us, for example, are cut off from our parents right now. I am, and I could be for months. Are our existing social networks adequate for facilitating communication with our older family members? Are they easy to use, do they offer the tools we need to help our parents and grandparents out in a time of crisis?
One immediate issue that came to my attention when all of this happened was that my parents need someone to bring supplies to them — if it is unsafe for them to venture outside, they will need someone to make sure they have food and toilet paper and soap and every other necessity. And for the very old among us there is the added concern of whether or not they are able to stay aware of their own needs.
I don’t personally feel like gaming or streaming entertainment is ripe for any newcomers as a result of Coronavirus. After all just about every type of media is available to stream these days — even comics! But I do think their importance in our lives is going to become much more evident now.
I have, for a while, started to think about ways that entrepreneurs can help consumers manage the growing number of subscriptions that are required now — either by way of bundles, or technologies to manage payment options, sharing, etc.
And finally, the big one: we need better technology to get information to the community.
And we don’t just need better channels — we need better noise reduction. The amount of utter nonsense I’ve seen on Twitter, Facebook and Instagram this week is depressing. I have had several friends and family members ask me if I had heard some tidbit working its way through the grapevine which was either comically false or, worse, potentially dangerous.
We need better ways to spread valid information than our current UGC platforms, which we have seen over the past few years are highly susceptible to misinformation and confusion.
I would personally love to see someone finally come in and solve not only the UGC news and information dissemination problem, but also make sure we have adequate tools in our pockets to receive and process updates from agencies like the CDC or WHO to keep us informed in the event of health crises or natural disasters.
That’s all for now. I’m sure over the next couple weeks or possibly months we’ll see more gaps in the market as a result of this.
One thing I would like to remind everyone is that the iPhone boom happened essentially right after the financial crisis of 2007-2008. And the just came out a ridiculously long period of growth since then. So stay positive, stay creative, and most importantly stay safe!
by OneEightHundred (noreply@blogger.com) at 2019-11-23 20:43
by OneEightHundred (noreply@blogger.com) at 2019-10-10 02:03
by OneEightHundred (noreply@blogger.com) at 2019-09-06 00:47
In the last post we were left with some tests that exercised some very basic functionality of the Deck
class. In this post, we will continue to add unit tests and write production code to make those tests pass, until we get a class which is able to produce a randomised deck of 52 cards.
You can, and should, refactor your tests where appropriate. For instance, on the last test in the last post, we only asserted that we could get all the cards for a particular suit. What about the other three? With most modern test frameworks, that is very easy.
[InlineData(Suit.Clubs)]
[InlineData(Suit.Diamonds)]
[InlineData(Suit.Hearts)]
[InlineData(Suit.Spades)]
public void Should_BeAbleToSelectSuitOfCardsFromDeck(Suit suit)
{
var deck = new Deck();
var cards = deck.Where(x => x.Suit == suit);
cards.Should().HaveCount(13);
}
We are going to want actual cards with values to work with. And for the next test, we can literally copy and past the previous test to use as a starter.
[Theory]
[InlineData(Suit.Clubs)]
[InlineData(Suit.Diamonds)]
[InlineData(Suit.Hearts)]
[InlineData(Suit.Spades)]
public void Should_BuildAllCardsInDeck(Suit suit)
{
var deck = new Deck();
var cards = deck.Where(x => x.Suit == suit);
cards.Should().Contain(new List<Card>
{
new Card(suit, "A"), new Card(suit, "2"), new Card(suit, "3"), new Card(suit, "4"),
new Card(suit, "5"), new Card(suit, "6"), new Card(suit, "7"), new Card(suit, "8"),
new Card(suit, "9"), new Card(suit, "10"), new Card(suit, "J"), new Card(suit, "Q"),
new Card(suit, "K")
});
}
Now that Iāve written this, when I compare it to the previous one, itās testing the exact same thing, in slightly more detail. So we can delete the previous test, itās just noise.
The test is currently failing because it canāt compile, due to there not being a constructor which takes a string. Lets fix that.
public struct Card
{
private Suit _suit;
private string _value;
public Card(Suit suit, string value)
{
_suit = suit;
_value = value;
}
public Suit Suit { get { return _suit; } }
public string Value { get { return _value; } }
public override string ToString()
{
return $"{Suit}";
}
}
There are a couple of changes to this class. Firstly, I added the constructor, and private variables which hold the two defining variables, with properties with only public getters. I changed it from being a class
to being a struct
, and itās now an immutable value type, which makes sense. In a deck of cards, there can, for example, only be one Ace of Spades.
These changes mean that are tests donāt work, as the Deck
class is now broken, because the code which builds set of thirteen cards for a given suit is broken - it now doesnāt understand the Card
constructor, or the fact that the .Suit
property is now read-only.
Here is my first attempt at fixing the code, which I donāt currently think is all that bad:
private string _ranks = "A23456789XJQK";
private List<Card> BuildSuit(Suit suit)
{
var cards = new List<Card>(_suitSize);
for (var i = 1; i <= _suitSize; i++)
{
var rank = _ranks[i-1].ToString();
var card = new Card(suit, rank);
cards.Add(card);
}
return cards;
}
This now builds us four suites of thirteen cards. I realised as I was writing the production code that handling ā10ā as a value would be straightforward, so I opted for the simpler (and common) approach of using āXā to represent ā10ā. The test pass four times, once for each suit. This is probably unnecessary, but it protects us in future from inadvertantly adding any code which may affect the way that cards are generated for a particular suit.
Itās occured to me as I write this that the Deck
class is funtionally complete, as it produces a deck of 52 cards when it is instantiated. You will however recall that we want a randomly shuffled deck of cards. If we consider, and invoke the Single Responsibility Principal, then we should add a Dealer
class; we are modeling a real world event and a pack of cards cannot shuffle itself, thatās what the dealer does.
In this post Iāve completed the walk through of developing a class to create a deck of 52 cards using some basic TDD techniques. I realised adding the ability to shuffle the pack to the Deck
class would be a violation of SRP, as the Deck
class should not be concerned or have any knowledge about how it is shuffled. In the next post I will discuss how we can implement a Dealer
class, and illustrate some techniques swapping the randomisation algorithim around.
by OneEightHundred (noreply@blogger.com) at 2018-03-30 05:26
In the previous post in this series, we had finished up with a very basic unit test, which didnāt really test much, which we had ran using dotnet xunit
in a console, and saw some lovely output.
Weāll continue to write some more unit tests to try and understand what kind of API we need in a class (or classes) which can help us satisfy the first rule of our Freecell engine implementation. As a reminder, our first rule is: There is one standard deck of cards, shuffled.
Iām trying to write both the code and the blog posts as I go along, so I have no idea what the final code will look like when Iāve finished. This means Iāll probably make mistakes and make some poor design decisions, but the whole point of TDD is that you can get a feel for that as you go along, because the tests will tell you.
Whilst we obey the 3 Laws of TDD, that doesnāt mean that we canāt or shouldnāt doodle a design and some notes on a whiteboard or a notebook about the way our API could look. I always find that having some idea of where you want to go and what you want to achieve aids the TDD process, because then the unit tests should kick in and youāll get a feel for whether things are going well or the conceptual design you had is not working.
With that in mind, we know that we will want to define a Card
object, and that there are going to be four suits of cards, so that gives us a hint that weāll need an enum
to define them. Unless we want to play the same game of Freecell over and over again, then weāll need to randomly generate the cards in the deck. We also know that we will need to iterate over the deck when it comes to building the Cascades, but the Deck
should not be concerned with that.
With that in mind, we can start writing some more tests.
Deck
classFirst things first, I think that I really like the idea of having the Deck
class enumerable, so Iāll start with testing that.
[Fact]
public void Should_BeAbleToEnumerateCards()
{
foreach (var card in new Deck())
{
}
}
This is enough to make the test fail, because the Deck
class doesnāt yet have a public definition for GetEnumerator
, but it gives us a feel for how the class is going to be used. To make the test pass, we can do the simplest thing to make the compiler happy, and give the Deck
class a GetEnumerator
definition.
public IEnumerator<object> GetEnumerator()
{
return Enumerable.Empty<object>().GetEnumerator();
}
Iām using the generic type of object
in the method, because I havenāt yet decided on what that type is going to be, because to do so would violate the three rules of TDD, and it hasnāt yet been necessary.
Now that we can enumerate the Deck
class, we can start making things a little more interesting. Given that it is a deck of cards, it should be reasonable to expect that we could expect to be able to select a suit of cards from the deck and get a collection which has 13 cards in it. Remember, we only need to write as much of this next test as is sufficient to get the test to fail.
[Fact]
public void Should_BeAbleToSelectSuitOfCardsFromDeck()
{
var deck = new Deck();
var hearts = deck.Where();
}
It turns out we canāt even get to the point in the test of asserting something because we get a compiler failure. The compiler canāt find a method or extension method for Where
. But, the previous test where we enumerate the Deck
in a foreach
passes. Well, we only wrote as much code to make that test pass as we needed to, and that only involved adding the GetEnumerator
method to the class. We need to write more code to get this current test to pass, such that we can keep the previous test passing too.
This is easy to do by implementing IEnumerable<>
on the Deck
class:
public class Deck : IEnumerable<object>
{
public IEnumerator<object> GetEnumerator()
{
foreach (var card in _cards)
{
yield return card;
}
}
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
Iāve cut some of the other code out of the class so that you can see just the detail of the implementation. The second explicitly implemented IEnumerable.GetEnumerator
is there because IEnumerable<>
inherits from it, so it must be implemented, but as you can see, we can just fastward to the genericly implemented method. With that done, we can now add using System.Linq;
to the Deck
class so that we can use the Where
method.
var deck = new Deck();
var hearts = deck.Where(x => x.Suit == Suit.Hearts);
This is where the implementation is going to start getting a little more complicated that the actual tests. Obviously in order to make the test pass, we need to add an actual Card
class and give it a property which can use to select the correct suit of cards.
public enum Suit
{
Clubs,
Diamonds,
Hearts,
Spades
}
public class Card
{
public Suit Suit { get; set; }
}
After writing this, we can then change the enumerable implementation in the Deck
class to public class Deck : IEnumerable<Deck>
, and the test will now compile. Now we can actually assert the intent of the test:
[Fact]
public void Should_BeAbleToSelectSuitOfCardsFromDeck()
{
var deck = new Deck();
var hearts = deck.Select(x => x.Suit == Suit.Hearts);
hearts.Should().HaveCount(13);
}
In this post, I talked through several iterations of the TDD loop, based on the 3 Rules of TDD, in some detail. An interesting discussion that always rears its head at this point is: Do you need to follow the 3 rules so excruciatingly religously? I donāt really know the answer to that. Certainly I always had it in my head that I would need a Card
class, and that would necessitate a Suit
enum, as these are pretty obvious things when thinking about the concept of a class which models a deck of cards. Could I have taken a short cut, written everything and then wrote the tests to test the implementation (as it stands)? Probably, for something so trivial.
In the next post, I will write some more tests to continue building the Deck
class.
Lock’n’Roll, a Pidgin plugin for Windows designed to set an away status message when the PC is locked, has received its first update in three and a half years!
Daniel Laberge has forked the project and released a version 1.2 update which allows you to specify which status should be set when the workstation locks. Get it while it’s awesome (always)!
terVec3 lb = ti->points[1] - ti->points[0];
terVec3 lc = ti->points[2] - ti->points[0];
terVec2 lbt = ti->texCoords[1] - ti->texCoords[0];
terVec2 lct = ti->texCoords[2] - ti->texCoords[0];
// Generate local space for the triangle plane
terVec3 localX = lb.Normalize2();
terVec3 localZ = lb.Cross(lc).Normalize2();
terVec3 localY = localX.Cross(localZ).Normalize2();
// Determine X/Y vectors in local space
float plbx = lb.DotProduct(localX);
terVec2 plc = terVec2(lc.DotProduct(localX), lc.DotProduct(localY));
terVec2 tsvS, tsvT;
tsvS[0] = lbt[0] / plbx;
tsvS[1] = (lct[0] - tsvS[0]*plc[0]) / plc[1];
tsvT[0] = lbt[1] / plbx;
tsvT[1] = (lct[1] - tsvT[0]*plc[0]) / plc[1];
ti->svec = (localX*tsvS[0] + localY*tsvS[1]).Normalize2();
ti->tvec = (localX*tsvT[0] + localY*tsvT[1]).Normalize2();
by OneEightHundred (noreply@blogger.com) at 2012-01-08 00:23