Introduction

AI and Machine Learning instruments such as libraries, available pre-trained models and open source datasets and data annotation platforms have advanced so much in recent years that within a small period of time and effort it is already possible to develop useful systems that automate business processes. The fact that large companies are exposing the results of their research and development to the public such as OpenAI’s GPT-3, even if it is paid, accelerates the process of creating end-user products even further. Such APIs are not cheap, but considering the costs that are needed for research and development of such models from scratch, the price is more than justified. For example, the one-time training of GPT-3 from the ground up alone is 12 million dollars of computational time:

So if one needs the impressive results GTP-3 can deliver they should be ready to pay for existing APIs.

Since the world is not black and white, one is not choosing between very expensive, but perfect quality, and low price (free), but bad quality. It is also possible to achieve good enough results for solving real business problems with limited expenses by combining open source solutions with the investment in creation of proprietary datasets and being more flexible with implementation timelines.

Now let’s discuss a practical example of a project from our experience with one of our customers.

We were approached by a client who was getting resumes of thousands of candidates’ every day on his platform with a request to automate some tasks using ML (machine learning) methods. One of the tasks was to automatically determine the seniority-level of the person’s position (job title). Since several thousand resumes pass through the client’s platform every day, and on average, each resume contains 2–3 positions, the total volume of job titles that need to be analysed daily is a lot, and therefore, the use of automation tools is fully justified, since solving such a problem with the involvement of a human resource becomes very time-consuming and expensive.

Below are the steps we took to solve the problem.

In total, 50 thousand job titles were annotated with the involvement of 11 people. We did it in 2 weeks and ended up at 40 seconds per job title speed.

In order to get started with Doccano, check their documentation.

Building a machine learning model with FastText

Our next step was the development of the ML model. For rapid prototyping, it is convenient to use the FastText framework (FaceBook) which we did. With their recently announced automatic hyperparameter tuning mechanism, you can quickly build very strong and high-quality baselines for text classification problems with very little effort.

To train a text classification model using FastText, you simply need to do the steps described bellow:

1. Divide data into train, test, valid sets

from sklearn.model_selection import train_test_split

train, test = train_test_split(data, test_size=0.15,

stratify=data['category'].values)

train, valid = train_test_split(train, test_size=0.15,

 stratify=train['category'].values)

2. Save train and test sets in FastText format on the disc. The format looks like this:

Data Scientist @ Medium __label__OtherVP of Sales at Microsoft __label__VP

3. Train a FastText model on train and valid sets using hyperparameter auto-tuning mechanism:

import fasttext
model = fasttext.train_supervised(input='job_level.train', autotuneValidationFile='job_level.valid', autotuneModelSize='2M')

4. Assess the quality of the resulting model by calculating metrics using the test part of the data:

import numpy as np
from sklean.metrics import classification_report
labels, probs = model.predict(test['title'].values)
y_pred = np.array(labels).ravel()
print(classification_report(test['category'].values, y_pred))

Model deployment using BentoML

The logical next step is the “deployment” of the trained model to the customer’s environment. We used the open source BentoML framework.

BentoML supports many ML frameworks, including, but not limited to:

• Scikit-Learn
• PyTorch
• Tensorflow 2
• FastText
• H2O
• Spacy
• Transformers

For the complete list, check the frameworks section of their documentation.

BentoML also allows having a trained model in the shortest possible time for delivering it with the required level of scalability. It supports model deployment in the following environments:

• AWS Lambda
• AWS SageMaker
• AWS EC2
• Azure Functions
• Docker
• Kubernetes

For more information about the deployment environments supported by BentoML, check their deployment guides.

The process for packaging and deploying a FastText model as a Docker container is the following:

1.Implement an inference service. The simplest example is shown bellow:

from bentoml import env, artifacts, BentoService, api
from bentoml.frameworks.fasttext import FasttextModelArtifact
from bentoml.adapters import JsonInput
@env(infer_pip_packages=True)
@artifacts([FasttextModelArtifact('model')])
class FasttextClassification(BentoService):
    
    @api(input=JsonInput(), batch=True)
    def predict(self, json_list):
        input = [i['text'] for i in json_list]
        result = self.artifacts.model.predict(input)
        # return top result
        prediction_result = [i[0].replace('__label__', '') for i in result[0]]
        return prediction_result

2. Save BentoService that contains the model:

from text_classification import FasttextClassification
svc = FasttextClassification()
svc.pack('model', model)

saved_path = svc.save()

3. Containerize model server with Docker:

bentoml containerize FasttextClassification:latest

Once packaging is finished, we have a docker image with our model wrapped with a RestAPI available, so we can run it locally or push it to the Docker-registry for further deployment to ECS, K8S, etc.

To test the API you can run it locally and check out the swagger docs:

bentoml serve {%saved_path%}