I often repeat myself, I know this. I am going to return to a thing I have blah’d on about more than once. The reason is that I think (yes me, that ego), there is a lot more to it than initially surmised.

In question we have my absolute fav, the Dunning-Kruger Effect, a scientific psychological proposal on human reasoning that states that people often believe of themselves that they have great knowledge on a subject simply because they have so little knowledge of that subject that they are unaware of how much they do not know. In other words, confidence often stands in the place of competence, but not as a deliberate action in many cases, mostly as a mistake. We can progress if we start to recognise this mistake in ourselves and others.

Now I will grant you that in many cases confidence is a deliberate action to fool the viewer into thinking that the speaker is an expert, a confidence trick, but this relies on the speaker guessing correctly that the audience that is listening are not experts on the subject, and that is often a gamble. In the case of people that know their audience is smarter on the subject there is always the use of force to rely on, like when Trump and Musk bloviate to a room full of hand picked journalists, or when the boss is again spouting some absolute nonsense about the business landscape to a room of nodding dogs. Sometimes confidence is fully deserved of course, it can be a result of actual expertise, this is where good arguments do not require power.

Dunning and Kruger proved (as much as a study can, I should really say that their work indicated, but that doesn’t sound so decisive), that people are very often the worst measuring stick for their own abilities. This phenomenon is not ubiquitous though, and it is not reserved for overestimations exclusively. What they found was that experts very often underestimate their expertise, and I think I have a theory on why that is. In the journey toward expertise the now person of expertise has occupied other positions on the graph (the DK) where they have been a non-expert, then a doubter, then gone through some sort of enlightenment and on to being said expert. There may be a cultivated doubtfulness lingering as an after effect of that path, the knowing that they cannot know it all.

I am no “expert” in any discipline, I have been described as an autodidact a few times by various people because I often learn things very quickly without formal study, though mainly only in a thematic sense. I have had many previous employment roles in a wide range of fields, am interested in investigating many subjects, I have studied many things, read many books, and of course I love a good conversation (a good conversation is not a monologue!). I may have gathered thematic data in many areas, but I am not an expert on anything I have ever done professionally or personally. I would say, if I had to say, that I’m a bit of an all-rounder, both in practical and intellectual terms, yet I also don’t mean I can turn my hand to anything. There are plenty of things I am really dreadful at regardless of effort or thinking. I like to describe myself as a fallible realist, and my life as a bullshit-free zone.

These days I happen to be an ICT Technician, though I never blog about computers because they are about as exciting to most people as a store bought cheese sandwich, and I am not an expert on them, I am a tech (so I am climbing the second peak toward expertise, occupying slot 4, see below graphic, but I will not reach slot 5 unless I specialise).

It’s just a tool, a PC, it does nothing if you do nothing with it, it’s no more a thing of wonder than a screwdriver, but it is more complex and a bit harder to understand. One illustration of the point I’m making on the DK scale is to take this tool, the PC, and to express how people use it/them based on their understanding of what it/they, and the wider discipline of computing in a business sense, does (what they think it can do and does).

Above is a rough drawing on a background I “borrowed” from the web. We can shoe horn the understanding of IT (Information Technology) or I&T (Information and Technology) into 5 categories in my estimation (and remember, I am not an expert on this subject, so I also may be in error when interpreting it). A person could occupy wildly different slots simultaneously based on their particular specialisations.

I must point out an important difference in the terms I am using, just so we are clear, IT and I&T are two differing things, the first is the technology that is used to convey, store, or process data (information), the second is how the technology is used when data is involved. Now that might not seem like much of a distinction, but it is. Separating I from T gets us thinking about the purpose rather than the software and hardware involved, it stops us being the sort of geeks that express the raw ability of the machine rather than the reason to use it.

If we concentrate on the I, set the T aside for now, and focus on data then we need to explain it. Data is that which can be measured, stored, and made useful to something else. It is the information about something, for example your height is an object of data, so is your current political leaning, so is your eye colour and so on. These are data objects when processed by data models, but information when a person needs to use them to make a decision. Now that we have gotten that out of the way, what has all this nonsense got to do with the DK?

Well, the DK does not relate to the data, or the technology, it is to do, in this situation, with the decisions that arise concerning data and the use of technology. I’m going to put people between the lines I have made, and give them the numbers that the segments represent.

Slot 1 – Users / operators / practitioners etc – these folks are in the technologically ignorant portion of the graph, but it doesn’t matter much because they use the tool for something in another discipline that they may hold greater expertise in, or they just perform a function they know how to do. What we do not want to see in here is a person who makes decisions on the usage of technology to process data or solve problems, we just want directed action within a defined mandate.

Slot 2 – Learners / inexperienced management candidates – people who are on the journey and are starting to realise that what once looked easy has become rather daunting. What we do not wish to see in this portion is decisions being made. Thematic postulate of the desired finishing point yes, the destination, but design of methodology no.

Slot 3 – Technicians and engineers of some calibre – here we have the people who express ability but with caution, ever engaging with higher levels of guiding expertise (often googling). Absorb desired thematic postulate on end result, participate as a component function toward building methodology to get to the desired destination, under the direction of an exert, or in some cases following a defined framework constructed by experts.

Slot 4 – Seasoned technicians and engineers – the people who can own the process and the decisions but remain cautious and engage with expertise in a collaborative way. Managers, supervisors and team leads fit here too if they are in the discipline, so do systems architects and the parts of management that direct the provision of technology to handle data, but only if they are capable! Importantly this is not cross discipline, each discipline can have it’s own DK Graph and I am only talking tech here in this example, it is hubris to think that because you are a Laundry manager that you can tailor service provision of technology for data processing, even if the data pertains to your department. The subject must be studied, not inherited, and cannot be granted.

Slot 5 – Experts – usually specialists in one or a couple of related I&T disciplines, rarely polymath but sometimes so. These are bearded sandal wearers who rarely see sunlight and scoff at folks that use visual IT tools if the command line is available. Expertise is focussed, it is not so that smart people are smart in every discipline, and it would be a mistake to assume so.

Okay, enough of my discipline, it truly is too boring to say anymore… Let’s look at the graphs from another perspective. Where would you put a GP or a Surgeon, a Pilot or a stewardess (do we still refer to them that way?), where does a plumber or a taxi driver go, where does a cleaner fit? A hairdresser? A Butcher? A Musician? I’ve got you thinking now, some of these distinctions are not clear because there are differences to how much they matter within their respective role. It does not matter how technically gifted a musician is if they make a good tune, none of the Beatles could read music and the Chuck Berry song You Never Can Tell has but two chords in it entirely.

Each is positioned by need, and constrained by ability, where there is a difference between where you would rightly expect them to be, and where they are, we have a problem if, and only if, they are lower than they are required to be concerning the need of their profession, or higher than they should be in regard to their credentials and remuneration. So if our plumber needed to be very good at plumbing then they have to occupy slots 2 or 3 for standard work, 4 or 5 for expertise and problem solving, but if they were an apprentice they would be constrained by lack of knowledge in the discipline and you would expect them to be in slot 1 or maybe 2. Now imagine our apprentice finds himself/herself project managing the installation of a heating system in a large industrial environment…. This has them doing labour that belongs in slot 4 and planning that belongs in slot 5, whilst they obviously inhabit slot 1 if they would then express the keenness to have a go without realising they lack the skill and knowledge to complete the job, and that is going to be a problem. It may not be a problem if things work out, it is not in every case that people are restrained by lack of knowledge or experience, but it is a good guide when considering if it is a good plan to engage them primarily.

This plumbing scenario is the essence of what the DK indicates, that it is in the not knowing, that something looks easier. It’s like being a consultant or the guy at the end of the bar, everything you say looks like you are saying it because you could do it, but that may, or may not, be just assumption. What you may state as the correct way to proceed may only look easy and correct to those that accept your description because they assume you have expertise and know that they do not. I always say that “everything looks easy when you don’t have to do it”, when you are merely looking at it from a point of ignorance and know that it will be on someone else to do the thing you are describing. Dentistry looks rather easy to me because I know nothing about it, but if I was wiser I would realise that dentists get a lot of training, so it cannot be that easy a thing to do. You get the point that it is far too easy to be dismissive or deflationary from behind the veil, and we should avoid that perspective?

The Trump/Musk hubris mistake…..

You run a business or you run a government department, you like some people so you engage them, say they have friends that need employment so you engage those guys also. You’re getting a nice little chumocracy going, built upon the idea that people will collaborate well together if they have already established a cohesive relationship. They seem like smart enough people to you, and that may work just fine if you’re running a chip shop or something equally as simple (I may be making the mistake I was warning against earlier, did you spot that?). The problem is this… your understanding of the remit you have granted them may be limited, as a result of your ignorance you may underestimate the difficulties involved. So they may all be in the wrong slots now because of the effect of your DK mistake, unless they all just happen to have expertise in the roles you have employed them for, which is wildly unlikely. If you pick people to assist you, and you base it on liking them, then I, and the DK, say you are making an error.

With the best of intentions a lack of awareness may build failed venture, create barriers to expertise, and put amateurism in the decision making parts of structure. Since decisions percolate down through structures to become actions in this case it likely ensures overall failure. The DK is exacerbated, it repeats for those you empower as it has for you. This failure can also be the case when replacing people of calibre with people who you favour simply because you favour them. Things may be in place that were built by competent labour, but the failure is merely delayed and it becomes harder to identify the source of in retrospect, the lagging. We are often fooled into thinking that what is happening now is the result of the governance of now, but we need to remember legacy in slow changing processes. Things change over long times, yet in differing disciplines they can change quite dramatically. In IT they change very very quickly, in Hairdressing they change at the speed of icebergs, or slower.

Efficiency and productivity matter, time is a resource that has a cost, decisions matter, structure and intellectual capital, or the absence of it, has impact. In markets, most ventures that go under do so because they have not been able to remain competitive in comparison with other ventures that do the same tasks or produce the same goods. The failures may have failed to change fast enough because they held a dogmatic and rigid approach coupled to a less than dynamic base of intellectual capital.

Cypher……

If we once again go back to my sphere (yawn), we see that the rise in cyber security investment by larger ventures is not accidental, it is a result of the rapid development of connective technology making data more available to those that need it, and that aspect comes with some danger. That rise presents a need for ever greater methods for the protection of data assets. As the technological landscape changes, any firm that continues to employ hubris or dogma, rather than realise this velocity and act accordingly, becomes more and more vulnerable to a potential compromise. This is just a fact in my discipline, that IT teams need to grow to encompass more specialisations, or the firm needs to invest in external intellectual capital in cyber security areas for I&T.

I’ll give you a real example… there was once a time when a simple cypher mechanism could obscure a data message so that virtually nobody could read it unless they had been told how to. This would have been based on an algorithm (a pattern, or a sequence of mathematics, or a code generated from a particular book etc), the Caesar Cipher is one of these. We take that sequence of encrypted data now and we put it into a computer program built for the task, in a matter of seconds to minutes the processing power of the computer, armed with the programming, accessing a database of all known previous patterns used by previously cracked ciphers as an analysis model, coupled with AI and distributed processing to enhance the horsepower, reveals both the pattern used and the data within. ChatGPT can crack the Caesar Cipher easily because it is a sequence based character shift within a single alphabet. In fact the Caesar Cipher may not even work well on humans anymore, especially those smart cookies that do game shows like Only Connect or the Krypton factor. This is because when it was first used people didn’t know what they were looking at (security through obscurity). Now we do cryptic crosswords, read Dan Brown novels, and play puzzle games on our tablets, so our compute engine is better at noticing tricky patterns and sequences, or at least realising they may be being employed.

It is naive to think that yesterday’s lock will keep today’s thief out, but that is often the case when firms, persons, and governments assess their data security stance using intellectual capital that does not realise it’s own lack of knowledge or expertise, like on the DK graph. I, as you, read almost daily about breaches and hacks that affect real businesses, but I, unlike you, mostly blame the management and not the IT professionals because these are not usually mere mistakes, or the product of IT staff laziness. I know, because I am in the discipline, that the firm’s IT department will likely be screaming for more resources in the form of qualified staff, more mandatory training, higher level security software, regular testing, greater capital spends engaging external entities, and most importantly good management of the department by a voice that is taken seriously from above. Often the vulnerability is not realised because it is not looked for by the team, but it is looked for by the hackers. So the firm has to find a way to act like the hacker does, before the hacker does. DK mistakes are what prevent this necessary action, that feeling of safety, of having done enough, spent enough, that ego that thinks it has expertise when it doesn’t, becomes the hackers’ advantage.

DK Mistakes……..

A mistake that can be made, and this is in line with the DK and technology, is to think that management can enable expertise through structural choice, that expertise itself is a granted or nurtured position, this is especially prevalent in IT matters as their methods are obscure and their stance can seem to limit access, so obstinate, but I would argue that this is for good purpose. Management can make good decisions yes, but more in line with the recognition of existing skills than in the creation of them through enablement, very few firms do their own IT training because it is not cost effective, more ordinarily the IT staff are externally trained, self trained, or bring expertise with them. A further mistake is to allow inaction to linger because the management feels the need to understand an issue rather than to trust the analysis of the technically able department in knowing what direction to take. It is also a potential DK mistake to overestimate individual attitude so as to think that those employees with the greatest emotional engagement will be the most effective in their discipline, as that is no indicator of ability (again, IT staff can seem obstinate). They may favour the employee that would willingly be a courtier rather than a professional, that doesn’t mean that they are not a professional though, but it is not the qualifying criteria of expertise or aptitude.

I was a soldier once, I memorised the principles of marksmanship, combat first aid, field craft etc, and could maybe recite them now twenty years later because they were useful in being a better soldier. They mattered, the words had practical applications to the day to day role. I could not tell you how many mission statements, vision statements, and branding documents I have encountered and then almost immediately forgotten within the last thirty years, because they had no bearing on the day to day in comparison to actual learned or gained experience. These statements are not expertise, they are intention on a trajectory, and they are the play thing of executives that may or may not have the necessary ability to turn them into something more than mere words. Saying, stating for all to know, that I intend to paint the shed, does not put a single droplet of paint on the shed.

In Conclusion…….

This is probably the only post I will ever do in relation to my my discipline, I apologise if it has bored you a bit, and it is important to reiterate yet again that I am no expert in anything I say regarding it other than thematically. The point was not to convey my job or to give insight into IT/I&T at all, it was to highlight something that can be very useful, an approach to thinking about confidence and expertise, I merely used what I do now as an example. I find the Dunning-Kruger examination to be a more than useful tool, one that can be applied to a multitude of disciplines in a multitude of settings, not merely the technical. Being aware of what you do not know may prove to be the single most important aspect in thinking about your approach to finding the most optimal solution to the problems faced.

The DK, a graph for everything.

Paul S Wilson