“Software is eating the world”.
When Marc Andresson opened his now famous blog post with this line back in 2011 he was predicting that software companies were about to take over large parts of the “traditional” economy. If we look at the explosive growth of software-powered products and companies in recent years it certainly seems like his prediction was correct. But what is software?
If hardware is our physical devices and their components, then software is the code, or instructions that tells the hardware what it needs to do. We can use your smartphone to explain the relationship. The latest high-end smartphones are hardware devices with high-quality camera lenses and sensor components. But these lenses and sensors are useless by themselves – if you want to film yourself dancing and post it to TikTok then we’ll also need software in the form of a smartphone operating system and camera software to capture and process our images and videos.
Note: These lessons are going to assume you have a basic understanding of computer hardware. If you need to brush up check out Technology 101: Hardware.
Let’s start by defining some important terms.
Software is a broad term for the collections of instructions (operating systems, programs and applications) that tell hardware devices how to perform tasks. Many of these tasks will be interactive and initiated by a user, like when you create a document on Microsoft Word or share a story on Instagram. The instructions, or code, that make up software are written by programmers and software developers using programming languages with names like Javascript, Python and Java.
Firmware is a special type of software that controls hardware, and doesn’t usually require any interaction with users. An example of firmware is the code on your smartphone that controls the initial loading sequence when you turn it on.
A program is a set of instructions that tells a hardware device what to do.
The operating system acts as the interface between hardware and the user, and is probably the most important category of software on a device. The operating system is in charge of managing the hardware and all of the other programs and applications that run on the device, from the calculator app through to the “device drivers” that let your computer communicate with your printer.
The most popular Desktop OS (an operating system for personal computers and laptops) are Microsoft Windows, Apple MacOS and Linux. The most popular smartphone operating systems are Google Android and Apple iOS. While each operating system acts as an interface between users and hardware, they have also been designed to prioritize certain features and functionality based on how they will be used. As an example, mobile operating systems like iOS and Android are very good at power management as mobile and tablet devices spend much of their life operating on battery power.
An application, or app, is a program that is designed to let users carry out specific tasks. An application can’t run on hardware by itself, it needs to interact with an operating system. There are a few types of applications, including:
Desktop applications are stored and run locally, or “natively”, on a standalone computer, performing specific tasks for the user. A common example of a desktop app would be the calculator app on your computer or laptop. Because desktop apps are run natively on the device they generally have access to more of the underlying hardware, which gives them greater performance capabilities than (for example) web applications. This is one of the main reasons why applications that require a lot of hardware resources to maximize performance, like video editing software or the latest games, tend to be desktop applications.
Web applications are accessed via a web browser like Chrome or Edge. Rather than being stored locally on the hardware device, the software is sent to the user’s browser from a remote server when it is needed. A popular example of a web app is Google Docs, a free word processing app that is hosted on Google’s servers and then loaded on your browser when you visit https://docs.google.com. While your browser tab is open you can use the app, but if you close the browser tab or lose your internet connection you will generally lose access to the application.
Simply letting your computer run any random code it comes across as you browse around would be a security nightmare. The major web browsers solve this problem by only letting code execute in special isolated environments called sandboxes. While this browser sandbox is secure it means the web app’s code does not have direct access to the device hardware, which limits its performance compared to a native app. When we use web applications we are making a tradeoff between convenience, security and performance.
Mobile applications: Mobile apps are stored and run locally, or natively, on your mobile device. Most of the time these apps are downloaded from Google or Apple’s app stores. Mobile apps are stored on the device itself, similar to desktop apps, so you won’t lose access to them when you close your browser tab.
But unlike most desktop apps, Android and iOS apps are also run in a security sandbox. The main difference between a web browser’s sandbox and a mobile sandbox is that the mobile application sandbox has been designed for specific operating systems and devices (like Apple’s iOS running on an Apple iPhone), which lets mobile application software developers write code that is optimized to take advantage of the capabilities of each device. In contrast, web browsers run on a wide range of devices of varying performance and capabilities so web application developers need to write code that can perform across all of them, whether it’s a powerful gaming PC or a 10 year old work laptop.
Programming
Programming is the act of writing the source code, or instructions, that make up software. There are many different programming languages that programmers can use when writing code. The language they choose to use will depend on a few factors, like the platform where the software will be used (desktop, mobile, web etc), whether speed and performance are a priority, or in many cases, the programmer’s personal preference.
Contrary to many online arguments, no single programming language is “best” but languages are often well suited to particular use cases.
Machine code is the language used to control a computer’s CPU. It’s designed to run as fast as possible for maximum performance and efficiency, and is written in a way that is readable for the computer rather than by humans. While it is technically possible to write in machine code it would be so time consuming and potentially error-prone that virtually no programmers actually do it – they rely on more abstracted, human-like languages.
High level programming languages like Javascript, Python or Ruby have been designed to be much closer to natural human language. This makes it much easier for programmers to write the code, and to understand code other programmers have written. High-level languages will usually hide or automate a lot of potentially complex processes, like how the system’s memory should be managed. This lets programmers focus on their core work (for example, creating a web app), rather than also having to spend time instructing the device how it should work.
Low level languages like C are closer to the native language of the computer, which makes them harder to understand than higher level languages. But using a low level language can allow programmers to write more efficient code, and have more control of how the computer system works. While it generally doesn’t matter if a simple web page isn’t highly optimized, when you’re writing the code for something computationally complex, like the graphics engine for a video game, then any extra speed or performance you can get from a low level language becomes very helpful.
The computer understands one sort of language (like machine code) but we’re programming in higher level languages, so how does that work? This is where compiling comes in. A compiler is a special type of computer program that takes source code from one language and automatically translates it into another language.
Programming languages in use
As I mentioned earlier, the choice of programming language will often come down to what you are trying to create. If you’re building apps for the web then you’ll almost certainly be using Javascript, as it’s understood by all of the major web browsers and used in most modern websites and web apps. If you’re working with large amounts of data then you might use Python as it has very good mathematical libraries (code that has been written to perform a specific task, and then shared for others to reuse in their own code). If you’re creating apps for iOS then you might use Apple’s Swift language, and if you’re building Android apps then could use Java or Kotlin.
Frameworks
In my free time I like to built web apps using Ruby on Rails. Ruby is a high-level programming language that is quite similar to Python, and the “on Rails” is the name for a framework that has been created that uses Ruby as the programming language. This framework helps speed up development by providing software developers with tools and common components they need to create a web app. As an example, Ruby on Rails comes with a built in web server that I can use, rather than trying and create a new web server myself.
Most popular programming languages will have frameworks that can be used to help speed up development. Some popular examples are ASP.NET Core, React.js, Vue.js and Django.
Algorithms & Machine learning
An algorithm is a set of rules that are followed to solve a problem or complete a task. Algorithm exist beyond the world of computing – if you follow a recipe to mix a great martini (and I’m sorry Bond but that means stirred with gin, not shaken with vodka), then you are using an algorithm.
We can break the process down into the Inputs, algorithm and Outputs. Using the Martini example above, the inputs would be the ingredients that we are using to make the drink. The algorithm is the set of rules we follow to mix the drink – “Measure and pour ingredients into a mixing glass. Stir the ingredients” etc. Since we know which ingredients we are using and we are following an algorithm (the recipe) to make the drink, we should be able to predict the output – in this case, a gin Martini.
Let’s bring this back to technology and computing. A common computing task is to sort a list of data into ascending or descending order, for example taking a list of exam results and sorting them from the highest to lowest grade. There are many sorting algorithms that are used to sort large lists in very efficient ways, and each is a variation of taking a list of data as the input, applying a sort algorithm, and then outputting the sorted data as the result.
It’s important to keep in mind that computers are incredibly powerful, but they are not intelligent in the same way as humans. While they are great at following well laid out instructions or algorithms, that’s exactly what they will do – they’ll follow the instructions as they have been programmed. Many things you would take for granted when speaking with a person will need to be explicitly instructed to a computer.
Let’s use the example of the Martini again. The first instruction in our recipe is “Measure and pour ingredients into a mixing glass”. What does measure mean? You’d probably guess (correctly) that I mean the amount of liquid we need for the recipe, but a computer won’t automatically know that – we could be asking it to find the length of the bottle or the distance between the bottle and the mixing glass.
This is why it’s so important for programmers to think logically and break everything down into small steps when they are writing code. If they don’t then they are liable to run into bugs and issues when the software executes exactly as it was written, rather than how it was intended.
Machine learning algorithms
Machine learning algorithms are a special type of algorithm that use data to make predictions or decisions. There are many different types of machine learning algorithms, but as an example a supervised learning algorithm will use a set of training data with known inputs and outputs to create a machine learning model (algorithm). This model can then be applied to future data where the inputs are known and we are trying to predict the outputs.
We can use the example of predicting house prices to explain this process. We start with our training data which includes inputs like the size of the houses, location, and the number of bedrooms and bathrooms. Since this is training data we also know the outputs, which in this case is the price each house most recently sold for. We can use our training data to build a machine learning model, and once this is ready we can apply it to new (non-training) input data to predict the price of a house.
Databases
A database is an organized collection of stored data. Or in other words, it’s a place where data is stored so it can be accessed and used at a later time. Databases are a fundamental part of the technology world, and almost every software product, web site and app makes use of them.
So how do they work? Most databases use tables, columns and rows to store information, kind of like a giant spreadsheet. Each table can represent a collection of data (like Users or Music), and these tables can have relationships with each other. Let’s look at an example.
When you open up Instagram the app makes a number of requests to Instagram’s servers, which then query Instagram’s database (or more realistically, databases) for data. While I can’t claim to know the exact structure of their systems, for the purpose of our example we will assume it’s similar to the simplified design I’m outlining below.
Their database will have a User table, which holds key information about Instagram’s users. The table will have columns for each user attribute: the things we’re using to define each user, like a unique ID, their username, email address and password. Each row in the user table contains the information for one user.
An application like Instagram will have databases with millions, or even billions of rows of data.
Instagram will have other tables in their database to store other important information, like records of the photos that have been uploaded to the service.
If you look in the Photo Table screenshot above, you’ll see that it has a “User ID” column, just like the User Table. If you look very closely you’ll see that some of the User IDs in each table match. This is an example of how we can form a relationship between the two database tables – we are basically saying “This photo in the database belongs to the user with ID 1234”.
When you open up your profile screen on Instagram the server knows it needs to grab all of the photos you’ve posted, so they do a query (a kind of search) on their database to look for all of the photos that are related to you. In a relational database this query will usually be written in a special language called SQL (Structured Query Language), so some of the most popular databases have names that include this acronym, like MySQL and PostgreSQL. If you end up in a technical role like programming or data science then you’ll be working with databases, so it would be worth taking some time to learn the basics of SQL.
Open Source vs Closed Source software
f I asked you to name some examples of software you use often you’d probably reply with an app you use at work (Google Docs or one of the Microsoft Office tools), to pass a few spare minutes (Twitter, Instagram or TikTok) or to relax after a long day (Netflix). All of the software examples I’ve just mentioned are known as closed source or proprietary software. The name “closed source” relates to the software’s source code – it’s “closed” in the sense that it is not shared publicly. This code is a valuable business asset and probably contains commercially valuable information so the owners do not want it to be publicly accessible. Closed source software is normally governed by a software license that lets you use the application, while the copyright holder (the person or company that made the software) maintains all ownership rights to the code. It might surprise you to know that in many cases when you buy a software product, even in a one-off purchase, you are technically only buying a license to use the software, and if you read the Terms of Service there may be some “interesting” restrictions on how you are legally allowed to use it.
But not all software is closed. Open Source Software (OSS, commonly referred to as open source) is a category of computer software where the copyright holder has decided to give other people the right to use, modify and distribute the source code and software to anyone else, for any purpose. Much of the underlying technology powering the internet is open source, for example a large percentage of web servers run versions of the open source Linux operating system, the open source content management platform WordPress powers millions of websites, and developers write code using Microsoft’s open source Visual Studio Code text editor.
With the rise of platforms like Github (a site that stores source code online for backup and shared access), it has become easy for developers to collaborate with each other to write software. There are hundreds of thousands of open source software projects hosted on Github, and collaboration is usually open to anyone who is willing to contribute – as long as the code that they write is good enough.
But why would anyone (much less a for-profit company like Microsoft) want to give away their work for free? There are actually quite a few reasons:
- Less time spent “reinventing the wheel”
- When people and companies can share resources it means they don’t have to keep remaking versions of the same basic software tools. For example, if a company wants to create a blog they don’t have to hire software engineers to build one from scratch, they can just use WordPress. This lets their developers spend more time working on high-value problems that provide real value to the business.
- More eyes to find and fix bugs
- All software has bugs, so the more people there are checking the code the greater the odds are of finding them before they cause problems.
- Philosophy – it’s good to help others and contribute to the community
- A lot of open source software is provided because the developers who write the code have a strong belief that open source software benefits their community.
- Good source of brand advertising
- Working on open source can help build positive brand associations for companies, especially in the eyes of developers. A recent example is Microsoft, who have received much more positive attention in the development community since they started releasing popular open source products like Visual Studio Code.
- Helps companies with hiring
- Developers like to work on interesting problems and contribute back to the community, so if they see a company contributing to open source projects they may be more inclined to work with them.
- Helps developers get hired
- And finally, but probably most importantly for those who are interested in working in a technical role – working on open source software can give you a great portfolio of publicly viewable work that you can share with potential employers. In fact, many employers will be more interested in speaking to a developer with open source contributions than with one who has completed some tech-related courses at university, which is great news if you are coming to tech from a non-traditional background.
User Interfaces, CLI & GUI
We’ve been spending some time talking about software in an abstract way, and now it’s time to look at it from the perspective of the user. A natural place to start is the User Interface.
The User Interface, or UI, is the place where humans and machines interact. The goal is to allow people to interact with these machines (computers, smartphones etc) in a simple, predictable way that allows them to achieve some sort of desired result.
To interact with machines we use input devices like a keyboard or mouse, and output devices like your computer’s monitor, a smartphone’s screen, speakers or a printer.
Command Line Interface (CLI)
Early computing devices used a Command Line Interface (CLI) where the user types in commands (instructions) as lines of text to interact with the computer. This kind of interaction is linear – the user types in a command and the machine responds. CLIs have a steep learning curve as you need to know a large number of commands to be able to interact with the computer.
Command line interfaces haven’t disappeared, although they are hidden out of sight of regular users. Many developers and technical users still choose to use the CLIs that are included in modern operating systems as they allow for complex interaction with the software. It’s kind of similar to modern cars – while most of us are happy to simply drive around and occasionally glance at the dashboard to check our gas or electricity levels, a mechanic still needs to be able to get into the engine bay and fix any issues that may come along.
Graphical User Interface (GUI)
As display technology improved to the level where they could start to display graphics (images), and eventually, colors, we started to move from command lines to the Graphical User Interface (GUI). GUIs use a mix of text, graphics, pointers (using a mouse or touchpad), and windows to convey information to a user. This information is generally shown in the form of windows, pages or screens, where each contains a mix of text, images, and interactive elements like buttons or links.
The move to GUIs was a major contributor to the growth in personal computers in the 1980s, as it was suddenly easy for non-technical users to use a PC. Apple’s Macintosh was the first commercially successful GUI-based computer, and the rest of the PC world followed soon after when Microsoft released their GUI-driven operating system, Windows.
More recently, smartphones and tablets moved our input device over from keyboards and mice to touchscreens, opening up a new set of gesture-based interactions like swiping, pulling and pinching.
Looking ahead
Some other common interfaces include Voice-controlled User Interfaces (VUI) like Apple’s Siri or Amazon’s Alexa that let us interact with our devices using speech. We also have video game consoles which use inputs like game controllers, and more recently, outputs like Virtual Reality (VR) headsets to interact with 3d virtual environments.
Application Programming Interface (API), JSON
If user interfaces let us with a computer, then how does a computer interact with another computer? Good question!
Computers and software use application programming interfaces, or APIs to connect with each other. An API is a way for a computer to make a request for data and then receive data back in a standardized form that it can understand.
We use APIs all the time – when you scroll through your Instagram feed the mobile app is making API calls (requests) to Instagram’s servers, asking for the details of new images and videos to display within the feed. Another common example is when you try to book a flight using a service like Expedia. After filling in your travel details (location, dates, number of people traveling etc) Expedia will make API calls at all the major airlines to see whether any seats are available on the selected days. The airlines will send back API responses to Expedia containing prices, departure times etc, who will use the information to generate a results page for you, containing all of the available flight options.
Another way to describe APIs is via our favorite analogy, the restaurant. Imagine you are sitting down looking at a menu deciding what to choose. The goal is to get your choice of food to the kitchen so they can make the meal. We could simply ask customers to go up to the window and start talking directly with the chefs and cooks, but that wouldn’t be very efficient. Instead, a waiter acts as an interface between the customer and the kitchen, taking the customer’s order (“I think I’d like a burger. Can you make sure it’s cooked through? And maybe take out the cheese. Also, would it be possible to add some mayonnaise?”) and presenting it to the kitchen staff in a systematic way (“Burger, well-done, no cheese, add mayo”). The waiter can then translate the kitchen’s response (“F$*@!!!”) back to the customer (“Yes, that will be fine.”).
Most APIs are either private (used to connect systems within a business) or public (available for anyone to use, for example a developer who wants to connect their app to another application).
When an API sends back a response it will commonly be in the form of a JSON (JavaScript Object Notation) file. JSON is an open stand file format that allows data to be transmitted in a structured, human-readable format. While it has “JavaScript” in the name, most modern programming languages are able to create and read JSON.
Next steps
If you need to brush up on your knowledge of hardware you can check out Technology 101: Hardware. If you’re ready to learn how tech companies and startups use technology to create products and make money visit Technology companies & startups.