Can you really “abstract away your database”?

TL;DR It’s possible to succeed for very narrow requirements and between a limited choice of databases. However it’s improbable you’ll succeed, and it’ll be late-night, high-scale edge cases that let you know.

“Just use a repository!”

Let me stop you there. If you use the word “just”, you’ve made a snap judgement and haven’t thought it through to all the possible branches of possibility.

First, a look at history

I personally believe this whole “abstract the database” thing came about properly in the late 1990’s when Oracle and DB2 were the RDBMS mainstays in “Enterprise”, and people were still writing “custom” databases to their own design of file structure. Along comes SQL Server, and suddenly there’s a new “Enterprise” option.

This is before the cloud. This is before “software as a service”. Even before enterprise apps were mostly web apps. It’s when you could say “enterprise” with a straight face. It’s probably before “enterprise” was really a thing as we know it now.

The landscape became such that any software you wrote that you delivered to your customer would have to run on their infrastructure, and you’d have little control over that. So if you wanted to cater for all customers, you needed to find a way to “configure” which database was in use.

However since your choices were between a small variety of RDMBS database engines, abstracting your database was, in theory, quite possible. It’s where Hibernate and Entity Framework came from. They are literally database abstractions.

Even SQL isn’t standard

ANSI SQL != T-SQL != PL/SQL and so on. All the “SQL” databases have either different syntaxes or different features or both. Here’s just a few differences I can think of (some have been aligned by now I expect, but I bet there’s still very little parity in the implementation):

  • SQL Server uses auto-incrementing IDENTITY columns. Oracle didn’t have those. You had to use a “sequence” and some PL/SQL to get auto-incrementing numbers. (I believe SQL Server now has sequences).
  • Datatypes are different. Some dbs have bools.
  • Oracle has Multi Version Concurrency Control. SQL Server didn’t (though there’s row versioning now, which is mostly the same).
  • The way that row locks are promoted to page locks and ultimately table locks may be different – so the same usage on two RDBMSs might suffer from deadlocking in one but not the other.
  • SQL Server participates in distributed transactions between databases and other DTC-capable things like MSMQ and (to a point RavenDB). Not all databases (i.e. non-Microsoft ones) do.

And that’s just RDBMSs. For example, GetEventStore doesn’t support transactions across two or more documents in it’s own database, let alone transactions across databases. Redis doesn’t even guarantee writes.

Urm, your abstraction is leaking…

Ultimately, a database abstraction is a leaky one at best. Your high-level code has to make assumptions about the implementation of lower-level code. Your interfaces don’t protect you here. If you’re “doing DDD” and/or CQRS for example, and you have your aggregate boundaries/transaction boundaries/context boundaries all set up nice, but you switch from SQL Server to Cassandra, you WILL eventually find that some assumptions about how data is written to and read from those systems are wrong. And it’ll happen at scale, when you least expect it, and it’ll be hard to debug.

So don’t kid yourself. And never say “just”.

Advertisements

Pragmatic Online Privacy – Passwords

Are you on Facebook? Do you Twitter? Perhaps Instagram is more your game? Do you shop with Amazon or order your groceries online?

If you have accounts with ANY online website, you have given away information about yourself to someone you don’t know, and you¬†cannot take it back. Even if you close your account, your data might be in backups, potentially for years afterwards. This is assuming you can be confident it never went any further in the first place…

This post is aimed at educating, rather than scaring, anyone computer literate enough to be reading it.

What is privacy?

Privacy is more than simple “cyber security”. It’s not about basic things like not disclosing your password or credit card PIN. It’s about considering what information you’re giving away.

Be a bit skeptical. Ask yourself why this app needs access to your contacts, why this Facebook quiz needs to be able to read your status updates, why this email is asking you to follow/click a hyperlink to sign-in (hint: no legitimate online site will EVER do this, so you can assume it’s a phishing attack if it happens to you. If you’re still unsure, contact the organisation directly by phone or by doing your own Google search for their homepage).

I’m not suggesting we throw the baby out with the bathwater and all leave the internet entirely. Just that we take a few simple precautions to avoid being “hacked”.

What is “hacking”?

Hacking isn’t always a nerd in a hoody in front of 9 computer screens with heaps of data flowing past as they break into some bank or top-secret government computer system, like in the movies.

That’s hard to do, for one thing.

It’s much easier to go after the weak link in any security system – the humans. I’ll do another post on “phishing”, but it’s basically the process of convincing a human to give away something about themselves that you can use to attack them with.

Imagine a Facebook quiz that will tell you which movie star you are most like in exchange for your mother’s maiden name and the name of your favourite pet as a child. Then, think back to what your backup security questions at the bank might be. Then ask yourself whether you know or trust the person who made that quiz.

Usernames and Passwords

This post is the first in a series, so let’s get the basics out of the way. There are simple rules to follow when dealing with credentials:

At the very very least, please, I beg you, ensure your email, online banking and “password manager” passwords are unique.

If an attacker successfully “hacks” your http://www.hondalovers.com account, they will immediately try those credentials on Amazon, all the banking websites, and all the email sites. Remember sometimes your username IS your email, which saves them time.

If they get into your email, they will change your password to prevent you intervening and set a computer program running that will go to all the other sites of value and try the “I forgot my password” option. This will send you them an email with the option to reset that password, which the attacker will make use of to lock you out of that account as well. Whatever that newly-compromised system is, chances are they know an order of magnitude more personal information about you, and can use that as further leverage on bigger, more “secure” targets.

Preferably, don’t even use the same password for more than one thing.

You might think it’s too many to keep track of, but recall the previous point where I described what they’ll do next once they get in. Is it worth the effort? I expect so.

There are tools (Dashlane, LastPass, 1Password) that can help with this. They can generate amazingly secure, unique, complex passwords for you, and even key them in for you when needed – even on your phone. Typically they are protected by a single “master” password. The rest of my advice here applies doubly so to that. However, these tools can cost money, so if you’re tight you don’t need the advanced features, you could use a password-protected note on your smartphone, for example.

Where you can’t use a password manager tool to manage and key-in wonderfully secure, unique passwords (often the case at work), then consider using the Correct Horse Battery Staple website to generate passwords that are easier to type in, and therefore “transfer” from your password manager app on your phone, to the system you’re signing into. Just remember that hackers take shortcuts, like trying all the words in the dictionary.

Don’t ever tell anyone your username and password for anything.

You might make arrangements in case “the worst” happens, so you don’t take your passwords with you, but generally speaking, never share your passwords with anyone, or share accounts. Most corporate IT policies make this mandatory, so think before you tell anyone “just use my account” that YOU ARE ACCOUNTABLE FOR ANY ACTION CARRIED OUT UNDER THAT ACCOUNT. Do you trust them that much?

Next time…

This post will become a series, with advice on considering your privacy in social media and on your smartphone. If you have any questions, please feel free to start a discussion in the comments section below. ūüôā

Alt.NET Never Went Away

“Two long posts in two days? What gives?”. Hello, Dear Reader. I’m late to the party, and those who know me know I usually have some sort of comment bursting to come out. Also, these things are important to me, my career, frankly my ability to put a roof over my childrens’ heads. I hope it makes vaguely interesting reading. Don’t worry, while I’ve decided to put more effort into blogging, I won’t be able to keep up this schedule. ūüôā

Alt.SAYWHAT?

Personally I was late and missed the movement that was Alt.NET. I eventually got¬†to .NET via¬†VB.NET in 2004 and C# a year or so later. When Alt.NET was “happening”, I was still newish and hadn’t yet outgrown the Microsoft-supplied default environment (although I had stopped installing the MSDN library from the disk). I had no idea there was a community. I’d never even heard of such a thing as a “user group”.

Alt.NET by Accident

However, despite not actually joining the Alt.NET movement, I still ended up looking outside the “out-of-the-box” experience eventually.¬†Yes I’ve used WinForms, SQL Server, DataSets, WPF, ASMX, WCF, WebForms (yuck), but I’ve also used NServiceBus, Dapper, GetEventStore, RavenDB, JavaScript. I use Vim as my text editor, and ReSharper+VsVim when I do use Visual Studio. When I did a spike with Neo4j a couple of years back, I installed Eclipse and used Java instead of bloody-mindedly sticking with the .NET client libraries. I’ve tried Rider and I like it. I’ve seen where I can do better than CRUD in places and implemented DDD, CQRS, Event Sourcing.¬†In personal projects I’ve had a bit of a go at Python and Node.js, among others.

I realise I’m sounding¬†massively big-headed here, while¬†trying to illustrate my breadth-firstedness. Allow¬†me balance that by admitting I’m not the world’s greatest programmer by a long chalk, but I believe in being aware of as much as possible and hopefully knowing things just well enough to know what, where, why, when and how¬†to look deeper¬†when necessary. That’s more difficult now than I’ve ever found it.

Bring Back Alt.NET?

While the Alt.NET movement may have passed into history, its values remain. Do we need to bring it back? I don’t think so. Should we remember it and refresh our collective memory, for our own benefit and the benefit of those coming into our industry? Yes, absolutely.

You see Alt.NET as a concept never died. It simply became the new status quo. People are now used to looking away from Microsoft for solutions. There is a problem that needs addressing, however. While the Alt.NET mindset is about looking outside for inspiration and bringing it back, or integrating it, the same does not appear to be the case from the other direction.

We need more balance.

The values we should be promoting these days should be inclusive. We don’t want .NET User Groups, Python User Groups, Java User Groups (is there even such a thing?) DevOps User Groups, ServerLess User Groups. We need Software Development User Groups. One big happy nerdy community of like-minded individuals sharing neat things. Whatever platform you’re on, don’t be afraid or too arrogant to look around and see what other platforms/languages/communities have on offer, and offer your own ideas. We’re not on opposite ends of a see-saw any more,¬†it’s a huge flat surface balanced on a point, and we’re spread out all over it. And besides, if we all came together on one point in the centre, it’d still balance.

Quick note on the health of .NET

I blogged about waning attitudes towards .NET already so I won’t revisit that here. The real question is: is .NET itself dying? My answer? I don’t know but I hope not and I suspect not overnight. I don’t believe the hyperbole of anti-Microsoft types, locked in their echo chamber, but I’m not naive enough to believe it’ll just live forever, and I recognise I’m in an echo chamber of my own, so I leave the door open and sit on the porch as often as possible.

So far I’ve only found one person¬†who’s bothered to actually try and analyse the situation. His results suggest .NET might be losing momentum “on the server”, hence the suggestion for a renaissance – a rejuvenation of sorts, but while interest might be waning, that doesn’t mean it’s any less capable.

.NET can make a huge comeback on the server with¬†cross-platform .NET Core, containerisation, serverless, the cloud, and since any problem can be solved with a layer of indirection, I suggest we leave the too-specific term “Alt.NET” itself behind, and stand behind a wider philosophy of inclusion, balance, humility. A kind of “Alt.DEV”, if you will.

Why so Anti-Microsoft Lately?

There is much negative feeling about Microsoft from some in the dev community of late. Either that or .NET Core just happens to have given them fresh ammunition for Twitter.

I suppose like many large¬†companies, they have over time made asshole moves in an effort to make as much money as possible. They have attempted to squash open source projects by releasing their own competitive efforts, knowing full well that the “dark matter developers” will likely adopt their solution, unaware¬†of the choices available to them (or simply content with what they’ve got). MS¬†could’ve invested effort in OpenRasta, but instead built WebAPI. They could’ve invested effort in OpenWrap but instead built nuget (I don’t mean to pick on Seb here; they’re just the examples that sprung to mind). Not to mention the recent¬†debugging library fiasco.

However it’s not all bad. I believe that they have made, and continue to produce, some excellent tech. Visual Studio is excellent. C# is a great language, powerful but very usable and easy to learn. MVC is good. WebAPI is good. Even Entity Framework 5+ is great now (but do yourself a favour and use the EF Reverse POCO Generator tool). Xaml is excellent and powerful (but much too verbose). Nuget is okay. Well, it solves what was a massive problem, and only introduces a little friction to pay for it. It’s adoption for other things, like Octopus Deploy, mean it’s actually quite useful.

On top of this, there are some great¬†libraries/frameworks built on .NET. NServiceBus is a particular favourite of mine, but there’s NancyFx, RavenDB, Json.NET, RestSharp.

.NET itself is and has been ideal for many things, both on the server and the desktop. That is not rendered false just because there is now more choice.

While I suppose I would be labelled as a .NET developer currently, I’m actually¬†interested in balance and open-mindedness, rather than outright¬†pushing MS. I don’t believe that equality means¬†overbalancing the scales the other way.

Some take the attitude that if you’re still on .NET, your career is dying. .NET itself is dying. .NET Core is failure. One person even suggested Windows has no future. There’s an attitude from some that one is¬†a bad developer and should use Go/Rust/Python/Scala/Java/LOLCODE. You should not use Windows. These people usually have some compelling war story or other, but rarely have anything objective¬†to back it up, and I’ve seen nothing that should induce panic ragequitting of the .NET platform.

.NET is merely one solution from many, and if you¬†decide against .NET for your project, do so not because you dislike the company, but because for your particular requirements it is not the best fit. Yes, .NET Core’s tooling progress has been a bit of a disaster while it works out what it wants to be. However, if you find yourself making architectural choices on a new project a year from now, you should be looking again at .NET Core to see if it’s right for you. Just because it’s not ready now doesn’t mean it won’t ever be, and you’d be remiss in your duties not to¬†bother finding out.

It’s not all about the language…

.NET and Microsoft is a PLATFORM. Learning new languages is easy. Learning their standard libraries is harder. Learning language¬†idioms is harder still. Learning how to build a scalable, robust app in your shiny new language that keeps it’s data in a store you’re not familiar with, on servers your team isn’t used to, and doing all that of that in an idiomatic way that won’t bite you in the ass in a few months’ time when its hard to change is a downright MAMMOTH task.

And it’s not all about you, either.

Remember if you have a team invested in .NET/Windows/Microsoft, that you can’t “just” start using Cassandra+RabbitMQ+Scala+Python+ReactJS on your next project. There is time needed for people to level-up on that tech – idioms learned, libraries chosen, spikes built and burned down. Perhaps new build tools to be purchased/deployed/understood/integrated/scripts written for. This is a big part of the consideration.

I’ve had people say of MS, .NET and related tech that “it’s a shitshow”, “there are better things”, “it’s dying”, “there’s no community”, “the libraries aren’t as good”, “there aren’t as many libraries”, “it’s legacy”. None of these statements really have anything to back them up, and one man’s magical solved-my-problem perfectly is another man’s OMG HOW DO EVEN TALK TO KAFKA WITH THIS? As a profession we’d do well to leave the prejudice behind and have some balance. Use what works and offers the best balance of learning/shiny and dependable/expedient.

Timerless Debounce in JavaScript

I wanted a debounce-like function today, but with a twist. Rather than waiting for a timeout, I wanted to wait until any current running instance of the function being debounced had finished.

Here’s a simple version of what I came up with:

function debounce2(func) {
    let isRunning = false;
    return function () {
        let context = this;
        let args = arguments;
        if (!isRunning) {
            isRunning = true;
            func.apply(context, args);
            isRunning = false;
        }
    };
};

As you can see, it’s just watching a boolean flag, that is set only while the function is running.

I needed support for Promises/Deferred, so my final version is this:

function debounce2(func) {
    let isRunning = false;
    let done = () => {
        isRunning = false;
    };
    return function () {
        let context = this;
        let args = arguments;
        if (!isRunning) {
            isRunning = true;
            let result = func.apply(context, args);
            if (result && result.hasOwnProperty("then")) {
                result.then(done, done);
            } else {
                isRunning = false;
            }
        }
    };
};

Hope this turns out to be useful. I’m not sure it’s technically a “debounce” or not, so if anyone has suggestions for a better name, please let me know in the comments.

LinkedIn is dead to me

Dear LinkedIn.

You don’t realise this, but your site is dying. It’s suffering from the social media equivalent of an auto-immune disease. The problem¬†is recruiters. Recruiters who hassle us and pretend they’re our friends or prior connections, when in fact they are not.

I no longer use LinkedIn because I can’t –¬†most of the activity is recruiters spamming me and the signal/noise ratio is impossible to overcome. I’m only keeping my account open so no-one takes over my name somehow.

The solution? Recruitment-specific functionality. Functionality that allows those of us not “open to new opportunities” to opt-out, and instead maintain a professional network with those we’ve actually done work with. Functionality that allows recruiters to actually connect with people who are looking for work. Keep that separate from the typical “network”, which ought to accurately represent my professional connections, not a bunch of lazy recruiters whom I’ve never done business with.

Yours regretfully,

A consternated user.

Testing URLs with PowerShell

Most people have heard of curl, but on Windows we haven’t always got that to hand. Well, the PowerShell team have thought of that.

If you just want to see whether a particular URL is working it’s as simple as:

$url = 'http://www.google.com/'
Invoke-WebRequest $url

They’ve even created curl as an alias for it. Neat.

Sometimes you’ll want to test against a web app on your intranet that requires credentials. That’s easy, too:

$url = 'http://www.google.com/'
$cred = Get-Credential 'domain\username' # Will show dialog asking for password.
curl $url -credential $cred

And of course you can perform all the usual lovely PowerShell piping greatness:

1,2,3 | %{ curl 'server$_.mydomain.com' }