How well should devs know how computers work?

Scott Hanselman asked this question on Twitter just now:

I wanted to reply but the Twitter character limit blah blah so here I am.

The short version is that it’s probably not necessary for most folks to really know the details of how computers do their thing, but given how even modern programming languages still leak some of the lower-level implementation (threads, garbage collection/allocation of memory etc) it probably helps. Especially if you ever have performance issues that need correcting. Could an entire team get away without knowing these details? Probably, at least until they needed to scale. Then it’s time to hire folks who have that deeper understanding.

To answer Scott’s questions more thoroughly:

How many of you are interested in “internals” of computers?

I put my hand up here. I think it’s probably a personal thing, though, I once asked on StackOverflow “Where and how does software meet hardware?”. It was closed 12 years after I wrote it (gotta love StackOverflow) but in essence it was asking how characters typed on a keyboard end up operating a processor. I got my answer in the end by reading the excellent CODE by Charles Petzold. I was fascinated to learn about Morse code and the beginnings of electronic telegraph systems.

Then on to how multiple electronic switches can be assembled to make logic gates. I already knew this, but never learned how logic gates could then be assembled to make latches to store single “bits” of data. How latches could be banked together (not always banks of 8 early on, either), how binary maths could then be applied to multiple banks of latches. Once you have this “computer memory”, you use some of that memory for data, some of it for storing programming instructions, some of it as an “instruction pointer” moved on by a “clock”… and so on and so on.

Do I need to know this to add an image tag to a webpage? No, but I find it fascinating as a subject in it’s own right, and programming is such a broad profession that this is almost a straw-man argument, because that’s not all everyone is doing.

Is it an “old person’s” game?

No, not at all. It’s not a matter of age, it’s a matter of how interested one is in the tools, and how that interest manifests itself. In Zen and the Art of Motorcycle Maintenance, the author categorises people as having either a “classical” or “romantic” mindset.

The “classical” way of thinking includes really understanding your machine. Getting enjoyment not only from using it, but from knowing how it works and even being able to perform maintenance on it for yourself.

The “romantic” mindset, on the other hand, treats the machine at face value. As an appliance. They don’t understand it well enough to maintain it themselves, but they use (and possibly enjoy) it for it’s intended purpose, and don’t even care that they couldn’t repair it for themselves.

In reality I believe this is likely correct, but with two additions:

• It depends on the subject. Someone might have a “classical” attitude towards old cars, and a “romantic” attitude towards modern cars.
• It’s probably not really mutually exclusive categories (life rarely works that way). In truth it’s more likely a sliding scale. One can have a classical attitude towards computers that only goes so far, and they’re happy to rely on the fact that every layer below those they care about works, without needing to know it for themselves.

Advertisement

Can you really “abstract away your database”?

TL;DR It’s possible to succeed for very narrow requirements and between a limited choice of databases. However it’s improbable you’ll succeed, and it’ll be late-night, high-scale edge cases that let you know.

“Just use a repository!”

Let me stop you there. If you use the word “just”, you’ve made a snap judgement and haven’t thought it through to all the possible branches of possibility.

First, a look at history

I personally believe this whole “abstract the database” thing came about properly in the late 1990’s when Oracle and DB2 were the RDBMS mainstays in “Enterprise”, and people were still writing “custom” databases to their own design of file structure. Along comes SQL Server, and suddenly there’s a new “Enterprise” option.

This is before the cloud. This is before “software as a service”. Even before enterprise apps were mostly web apps. It’s when you could say “enterprise” with a straight face. It’s probably before “enterprise” was really a thing as we know it now.

The landscape became such that any software you wrote that you delivered to your customer would have to run on their infrastructure, and you’d have little control over that. So if you wanted to cater for all customers, you needed to find a way to “configure” which database was in use.

However since your choices were between a small variety of RDMBS database engines, abstracting your database was, in theory, quite possible. It’s where Hibernate and Entity Framework came from. They are literally database abstractions.

Even SQL isn’t standard

ANSI SQL != T-SQL != PL/SQL and so on. All the “SQL” databases have either different syntaxes or different features or both. Here’s just a few differences I can think of (some have been aligned by now I expect, but I bet there’s still very little parity in the implementation):

• SQL Server uses auto-incrementing IDENTITY columns. Oracle didn’t have those. You had to use a “sequence” and some PL/SQL to get auto-incrementing numbers. (I believe SQL Server now has sequences).
• Datatypes are different. Some dbs have bools.
• Oracle has Multi Version Concurrency Control. SQL Server didn’t (though there’s row versioning now, which is mostly the same).
• The way that row locks are promoted to page locks and ultimately table locks may be different – so the same usage on two RDBMSs might suffer from deadlocking in one but not the other.
• SQL Server participates in distributed transactions between databases and other DTC-capable things like MSMQ and (to a point RavenDB). Not all databases (i.e. non-Microsoft ones) do.

And that’s just RDBMSs. For example, GetEventStore doesn’t support transactions across two or more documents in it’s own database, let alone transactions across databases. Redis doesn’t even guarantee writes.

Urm, your abstraction is leaking…

Ultimately, a database abstraction is a leaky one at best. Your high-level code has to make assumptions about the implementation of lower-level code. Your interfaces don’t protect you here. If you’re “doing DDD” and/or CQRS for example, and you have your aggregate boundaries/transaction boundaries/context boundaries all set up nice, but you switch from SQL Server to Cassandra, you WILL eventually find that some assumptions about how data is written to and read from those systems are wrong. And it’ll happen at scale, when you least expect it, and it’ll be hard to debug.

So don’t kid yourself. And never say “just”.

Pragmatic Online Privacy – Passwords

Are you on Facebook? Do you Twitter? Perhaps Instagram is more your game? Do you shop with Amazon or order your groceries online?

If you have accounts with ANY online website, you have given away information about yourself to someone you don’t know, and you cannot take it back. Even if you close your account, your data might be in backups, potentially for years afterwards. This is assuming you can be confident it never went any further in the first place…

This post is aimed at educating, rather than scaring, anyone computer literate enough to be reading it.

What is privacy?

Privacy is more than simple “cyber security”. It’s not about basic things like not disclosing your password or credit card PIN. It’s about considering what information you’re giving away.

Be a bit skeptical. Ask yourself why this app needs access to your contacts, why this Facebook quiz needs to be able to read your status updates, why this email is asking you to follow/click a hyperlink to sign-in (hint: no legitimate online site will EVER do this, so you can assume it’s a phishing attack if it happens to you. If you’re still unsure, contact the organisation directly by phone or by doing your own Google search for their homepage).

I’m not suggesting we throw the baby out with the bathwater and all leave the internet entirely. Just that we take a few simple precautions to avoid being “hacked”.

What is “hacking”?

Hacking isn’t always a nerd in a hoody in front of 9 computer screens with heaps of data flowing past as they break into some bank or top-secret government computer system, like in the movies.

That’s hard to do, for one thing.

It’s much easier to go after the weak link in any security system – the humans. I’ll do another post on “phishing”, but it’s basically the process of convincing a human to give away something about themselves that you can use to attack them with.

Imagine a Facebook quiz that will tell you which movie star you are most like in exchange for your mother’s maiden name and the name of your favourite pet as a child. Then, think back to what your backup security questions at the bank might be. Then ask yourself whether you know or trust the person who made that quiz.

Usernames and Passwords

This post is the first in a series, so let’s get the basics out of the way. There are simple rules to follow when dealing with credentials:

At the very very least, please, I beg you, ensure your email, online banking and “password manager” passwords are unique.

If an attacker successfully “hacks” your http://www.hondalovers.com account, they will immediately try those credentials on Amazon, all the banking websites, and all the email sites. Remember sometimes your username IS your email, which saves them time.

If they get into your email, they will change your password to prevent you intervening and set a computer program running that will go to all the other sites of value and try the “I forgot my password” option. This will send you them an email with the option to reset that password, which the attacker will make use of to lock you out of that account as well. Whatever that newly-compromised system is, chances are they know an order of magnitude more personal information about you, and can use that as further leverage on bigger, more “secure” targets.

Preferably, don’t even use the same password for more than one thing.

You might think it’s too many to keep track of, but recall the previous point where I described what they’ll do next once they get in. Is it worth the effort? I expect so.

There are tools (Dashlane, LastPass, 1Password) that can help with this. They can generate amazingly secure, unique, complex passwords for you, and even key them in for you when needed – even on your phone. Typically they are protected by a single “master” password. The rest of my advice here applies doubly so to that. However, these tools can cost money, so if you’re tight you don’t need the advanced features, you could use a password-protected note on your smartphone, for example.

Where you can’t use a password manager tool to manage and key-in wonderfully secure, unique passwords (often the case at work), then consider using the Correct Horse Battery Staple website to generate passwords that are easier to type in, and therefore “transfer” from your password manager app on your phone, to the system you’re signing into. Just remember that hackers take shortcuts, like trying all the words in the dictionary.

Don’t ever tell anyone your username and password for anything.

You might make arrangements in case “the worst” happens, so you don’t take your passwords with you, but generally speaking, never share your passwords with anyone, or share accounts. Most corporate IT policies make this mandatory, so think before you tell anyone “just use my account” that YOU ARE ACCOUNTABLE FOR ANY ACTION CARRIED OUT UNDER THAT ACCOUNT. Do you trust them that much?

Next time…

This post will become a series, with advice on considering your privacy in social media and on your smartphone. If you have any questions, please feel free to start a discussion in the comments section below. 🙂

Alt.NET Never Went Away

“Two long posts in two days? What gives?”. Hello, Dear Reader. I’m late to the party, and those who know me know I usually have some sort of comment bursting to come out. Also, these things are important to me, my career, frankly my ability to put a roof over my childrens’ heads. I hope it makes vaguely interesting reading. Don’t worry, while I’ve decided to put more effort into blogging, I won’t be able to keep up this schedule. 🙂

Alt.SAYWHAT?

Personally I was late and missed the movement that was Alt.NET. I eventually got to .NET via VB.NET in 2004 and C# a year or so later. When Alt.NET was “happening”, I was still newish and hadn’t yet outgrown the Microsoft-supplied default environment (although I had stopped installing the MSDN library from the disk). I had no idea there was a community. I’d never even heard of such a thing as a “user group”.

Alt.NET by Accident

However, despite not actually joining the Alt.NET movement, I still ended up looking outside the “out-of-the-box” experience eventually. Yes I’ve used WinForms, SQL Server, DataSets, WPF, ASMX, WCF, WebForms (yuck), but I’ve also used NServiceBus, Dapper, GetEventStore, RavenDB, JavaScript. I use Vim as my text editor, and ReSharper+VsVim when I do use Visual Studio. When I did a spike with Neo4j a couple of years back, I installed Eclipse and used Java instead of bloody-mindedly sticking with the .NET client libraries. I’ve tried Rider and I like it. I’ve seen where I can do better than CRUD in places and implemented DDD, CQRS, Event Sourcing. In personal projects I’ve had a bit of a go at Python and Node.js, among others.

I realise I’m sounding massively big-headed here, while trying to illustrate my breadth-firstedness. Allow me balance that by admitting I’m not the world’s greatest programmer by a long chalk, but I believe in being aware of as much as possible and hopefully knowing things just well enough to know what, where, why, when and how to look deeper when necessary. That’s more difficult now than I’ve ever found it.

Bring Back Alt.NET?

While the Alt.NET movement may have passed into history, its values remain. Do we need to bring it back? I don’t think so. Should we remember it and refresh our collective memory, for our own benefit and the benefit of those coming into our industry? Yes, absolutely.

You see Alt.NET as a concept never died. It simply became the new status quo. People are now used to looking away from Microsoft for solutions. There is a problem that needs addressing, however. While the Alt.NET mindset is about looking outside for inspiration and bringing it back, or integrating it, the same does not appear to be the case from the other direction.

We need more balance.

The values we should be promoting these days should be inclusive. We don’t want .NET User Groups, Python User Groups, Java User Groups (is there even such a thing?) DevOps User Groups, ServerLess User Groups. We need Software Development User Groups. One big happy nerdy community of like-minded individuals sharing neat things. Whatever platform you’re on, don’t be afraid or too arrogant to look around and see what other platforms/languages/communities have on offer, and offer your own ideas. We’re not on opposite ends of a see-saw any more, it’s a huge flat surface balanced on a point, and we’re spread out all over it. And besides, if we all came together on one point in the centre, it’d still balance.

Quick note on the health of .NET

I blogged about waning attitudes towards .NET already so I won’t revisit that here. The real question is: is .NET itself dying? My answer? I don’t know but I hope not and I suspect not overnight. I don’t believe the hyperbole of anti-Microsoft types, locked in their echo chamber, but I’m not naive enough to believe it’ll just live forever, and I recognise I’m in an echo chamber of my own, so I leave the door open and sit on the porch as often as possible.

So far I’ve only found one person who’s bothered to actually try and analyse the situation. His results suggest .NET might be losing momentum “on the server”, hence the suggestion for a renaissance – a rejuvenation of sorts, but while interest might be waning, that doesn’t mean it’s any less capable.

.NET can make a huge comeback on the server with cross-platform .NET Core, containerisation, serverless, the cloud, and since any problem can be solved with a layer of indirection, I suggest we leave the too-specific term “Alt.NET” itself behind, and stand behind a wider philosophy of inclusion, balance, humility. A kind of “Alt.DEV”, if you will.

Why so Anti-Microsoft Lately?

There is much negative feeling about Microsoft from some in the dev community of late. Either that or .NET Core just happens to have given them fresh ammunition for Twitter.

I suppose like many large companies, they have over time made asshole moves in an effort to make as much money as possible. They have attempted to squash open source projects by releasing their own competitive efforts, knowing full well that the “dark matter developers” will likely adopt their solution, unaware of the choices available to them (or simply content with what they’ve got). MS could’ve invested effort in OpenRasta, but instead built WebAPI. They could’ve invested effort in OpenWrap but instead built nuget (I don’t mean to pick on Seb here; they’re just the examples that sprung to mind). Not to mention the recent debugging library fiasco.

However it’s not all bad. I believe that they have made, and continue to produce, some excellent tech. Visual Studio is excellent. C# is a great language, powerful but very usable and easy to learn. MVC is good. WebAPI is good. Even Entity Framework 5+ is great now (but do yourself a favour and use the EF Reverse POCO Generator tool). Xaml is excellent and powerful (but much too verbose). Nuget is okay. Well, it solves what was a massive problem, and only introduces a little friction to pay for it. It’s adoption for other things, like Octopus Deploy, mean it’s actually quite useful.

On top of this, there are some great libraries/frameworks built on .NET. NServiceBus is a particular favourite of mine, but there’s NancyFx, RavenDB, Json.NET, RestSharp.

.NET itself is and has been ideal for many things, both on the server and the desktop. That is not rendered false just because there is now more choice.

While I suppose I would be labelled as a .NET developer currently, I’m actually interested in balance and open-mindedness, rather than outright pushing MS. I don’t believe that equality means overbalancing the scales the other way.

Some take the attitude that if you’re still on .NET, your career is dying. .NET itself is dying. .NET Core is failure. One person even suggested Windows has no future. There’s an attitude from some that one is a bad developer and should use Go/Rust/Python/Scala/Java/LOLCODE. You should not use Windows. These people usually have some compelling war story or other, but rarely have anything objective to back it up, and I’ve seen nothing that should induce panic ragequitting of the .NET platform.

.NET is merely one solution from many, and if you decide against .NET for your project, do so not because you dislike the company, but because for your particular requirements it is not the best fit. Yes, .NET Core’s tooling progress has been a bit of a disaster while it works out what it wants to be. However, if you find yourself making architectural choices on a new project a year from now, you should be looking again at .NET Core to see if it’s right for you. Just because it’s not ready now doesn’t mean it won’t ever be, and you’d be remiss in your duties not to bother finding out.

It’s not all about the language…

.NET and Microsoft is a PLATFORM. Learning new languages is easy. Learning their standard libraries is harder. Learning language idioms is harder still. Learning how to build a scalable, robust app in your shiny new language that keeps it’s data in a store you’re not familiar with, on servers your team isn’t used to, and doing all that of that in an idiomatic way that won’t bite you in the ass in a few months’ time when its hard to change is a downright MAMMOTH task.

And it’s not all about you, either.

Remember if you have a team invested in .NET/Windows/Microsoft, that you can’t “just” start using Cassandra+RabbitMQ+Scala+Python+ReactJS on your next project. There is time needed for people to level-up on that tech – idioms learned, libraries chosen, spikes built and burned down. Perhaps new build tools to be purchased/deployed/understood/integrated/scripts written for. This is a big part of the consideration.

I’ve had people say of MS, .NET and related tech that “it’s a shitshow”, “there are better things”, “it’s dying”, “there’s no community”, “the libraries aren’t as good”, “there aren’t as many libraries”, “it’s legacy”. None of these statements really have anything to back them up, and one man’s magical solved-my-problem perfectly is another man’s OMG HOW DO EVEN TALK TO KAFKA WITH THIS? As a profession we’d do well to leave the prejudice behind and have some balance. Use what works and offers the best balance of learning/shiny and dependable/expedient.

Timerless Debounce in JavaScript

I wanted a debounce-like function today, but with a twist. Rather than waiting for a timeout, I wanted to wait until any current running instance of the function being debounced had finished.

Here’s a simple version of what I came up with:

function debounce2(func) {
    let isRunning = false;
    return function () {
        let context = this;
        let args = arguments;
        if (!isRunning) {
            isRunning = true;
            func.apply(context, args);
            isRunning = false;
        }
    };
};

As you can see, it’s just watching a boolean flag, that is set only while the function is running.

I needed support for Promises/Deferred, so my final version is this:

function debounce2(func) {
    let isRunning = false;
    let done = () => {
        isRunning = false;
    };
    return function () {
        let context = this;
        let args = arguments;
        if (!isRunning) {
            isRunning = true;
            let result = func.apply(context, args);
            if (result && result.hasOwnProperty("then")) {
                result.then(done, done);
            } else {
                isRunning = false;
            }
        }
    };
};

Hope this turns out to be useful. I’m not sure it’s technically a “debounce” or not, so if anyone has suggestions for a better name, please let me know in the comments.

LinkedIn is dead to me

Dear LinkedIn.

You don’t realise this, but your site is dying. It’s suffering from the social media equivalent of an auto-immune disease. The problem is recruiters. Recruiters who hassle us and pretend they’re our friends or prior connections, when in fact they are not.

I no longer use LinkedIn because I can’t – most of the activity is recruiters spamming me and the signal/noise ratio is impossible to overcome. I’m only keeping my account open so no-one takes over my name somehow.

The solution? Recruitment-specific functionality. Functionality that allows those of us not “open to new opportunities” to opt-out, and instead maintain a professional network with those we’ve actually done work with. Functionality that allows recruiters to actually connect with people who are looking for work. Keep that separate from the typical “network”, which ought to accurately represent my professional connections, not a bunch of lazy recruiters whom I’ve never done business with.

Yours regretfully,

A consternated user.

Testing URLs with PowerShell

Most people have heard of curl, but on Windows we haven’t always got that to hand. Well, the PowerShell team have thought of that.

If you just want to see whether a particular URL is working it’s as simple as:

$url = 'http://www.google.com/'
Invoke-WebRequest $url

They’ve even created curl as an alias for it. Neat.

Sometimes you’ll want to test against a web app on your intranet that requires credentials. That’s easy, too:

$url = 'http://www.google.com/'
$cred = Get-Credential 'domain\username' # Will show dialog asking for password.
curl $url -credential $cred

And of course you can perform all the usual lovely PowerShell piping greatness:

1,2,3 | %{ curl 'server$_.mydomain.com' }

PowerShell: Take care with null arguments and SilentlyContinue…

I just had an excellent gotcha at work that took down one of our test servers.

I’m using PowerShell scripts as part of an msdeploy process to stop a service in the presync command so that the files can be updated, followed by yet more PowerShell starting the service back up again in the postsync command.

My script looked like this:

Get-Service $serviceName -ErrorAction SilentlyContinue | Stop-Service

I made two mistakes here and learned two valuable lessons.

Lesson 1 – Don’t misuse SilentlyContinue

For reasons I won’t bore you with, $serviceName wasn’t defined. Normally the interpreter would’ve halted proceedings and prevented anything from happening, but since I was eating all the errors it just went along and passed $null into Stop-Service. Which of course stopped all the services.

Yes, all of them.

Even Parental Controls. PARENTAL CONTROLS! THINK OF THE CHILDREN!

Since I wanted one specific service, I’ve now chosen to do something more like this:

$service = Get-Service | ?{ $_.ServiceName -eq $serviceName } | select -First 1

if ($service) {
    $service.Stop()
}

This isn’t very “functional” and PowerShell experts will have more elegant and PowerShell-like ways to resolve this, but at least I’m not eating exceptions any more.

Lesson 2 – Be careful with null arguments

In that first example, what if $serviceName was defined, but was currently null? Run this for yourself and see:

Get-Service $null

That’s right, no error, and it returns ALL your services. Which in my case would’ve been piped into Stop-Service…

This is obvious when you think about it. “Get-Service $null” is exactly the same as “Get-Service”. The default behaviour for which being to return all services.

So beware when using variables as arguments – think about what the default behaviour is if those values are null.

Lesson 3 – Scripts are software too

I said there were 2 lessons, but I just thought of this one so I threw it in. I should have known better than to eat all the exceptions. I’d never do this in C#, for example:

try
{
    // Do stuff
}
catch
{
    // Om nom nom
}

So why did I allow myself to do it here? I think subconsciously I was thinking “It’s only a script” but of course that was a mistake. In future I will try and make sure I treat scripts with the same respect as any other software I write. Even a simple batch file could do untold damage if badly written, let alone the more intricate things that are possible with PowerShell.

My first F# program

I love C# and have some experience in other languages. I have a hobby project and have contributed to another open source project as part of GiveCampUK. I love Pluralsight and although I’m still learning, am quite passionate about DDD, CQRS and Event Sourcing. However I still like to find alternative ways to exercise the “little grey cells” and so, despite having no use for it whatsoever, I’m going to make a concerted effort to learn F#. Here’s my first attempt of something marginally useful:

let pi = 3.141592654
let area r = pi * (r * r)
let x = area 2.0

I’ve been using tryfsharp.org, which seems to be quite a neat way to get into it. I’ll probably checkout Tekpub and Pluralsight, too. I just wish I knew what it was for…