Whereas statistical techniques often focus on learning more about a given dataset, machine learning focuses on finding complex relationships within the dataset and then applying it for predictive purposes. Machine learning techniques can already be applied to relatively small data sets, provided the right information is present within. Any kind of data can be used as input, but requires a conversion to a numerical form. Much of the difficulty of machine learning projects is found in constructing these inputs and finding the model best capable of picking the desired goal.
Feature construction is the discipline of translating the available inputs to a representation ideally suited for the machine learning model. The ideal representation strongly depends on the type of data, finding it is trivial in some projects, for others it may require careful thinking. For example, predicting sales from marketing data requires a much different approach than detecting objects in an image. Depending on the size and complexity of the data set, the number of inputs may also have to be greatly reduced to be able to train a model. For large data sets, with an abundance of useful information, deep learning may offer a valuable alternative.
Model selection is the second important step in the creation of a machine learning model. Many different machine learning techniques exist and the optimal choice of method may depend both on the input data and the task at hand. Imagine for instance the analysis of sales data. Results may depend on continuous variables such as the number of items sold, but also on categorical variables such as the type of product, the week day or the gender of the customer. Similarly one might try to predict a continuous variable, such as the sales volume of the next day or categorize or even cluster customers in groups to provide optimal targeting. Certain models are better at handling different types of input or output data. The availability and complexity of the data set is also a determining factor. Ideal prediction on a small data set may result from a simple linear model, whereas a complex data set may require advanced kernel methods to describe the complex patterns within.
Constructing the models described in the previous paragraphs is a process which relies heavily on human insight and intuition. While using human knowledge can greatly accelerate the creation of good models, it also introduces bias. Deep learning models offer a way to create complex models with less human interference. Given the right architecture of the model, this new class of models is capable of automatically constructing features for a given application. It can work with a wide variety of data inputs from spreadsheets to images, text and even graphs. The only caveat is that much larger data sets are needed to allow the model to decide which information is important for a given problem. Since many deep learning tasks are similar, however, existing data sets can often be used to facilitate the process. Various transformations can also be applied to in-house data to artificially expand them.
Want to learn how to apply machine learning in-house? We provide both beginner and advanced workshops tailored to your applications.
Want to bring machine learning to production? We can tie our deep learning models into existing data systems and even help you set up the necessary hardware.