Tutorial 1. Data Representation

Goals:

1. Learn how MLJ specifies it's data requirements using "scientific" types
2. Understand the options for representing tabular data
3. Learn how to inspect and fix the representation of data to meet MLJ requirements

To run the code in this tutorial in a live Julia session, first follow the instructions given here.

Scientific types

To help you focus on the intended purpose or interpretation of data, MLJ models specify data requirements using scientific types, instead of machine types. An example of a scientific type is OrderedFactor. The other basic "scalar" scientific types are illustrated below:

A scientific type is an ordinary Julia type (so it can be used for method dispatch, for example) but it usually has no instances. The scitype function is used to articulate MLJ's convention about how different machine types will be interpreted by MLJ models:

using ScientificTypes
scitype(3.14)

ScientificTypesBase.Continuous

time = [2.3, 4.5, 4.2, 1.8, 7.1]
scitype(time)

AbstractVector{Continuous} (alias for AbstractArray{ScientificTypesBase.Continuous, 1})

To fix data which MLJ is interpreting incorrectly, we use the coerce method:

height = [185, 153, 163, 114, 180]
scitype(height)

AbstractVector{Count} (alias for AbstractArray{ScientificTypesBase.Count, 1})

height = coerce(height, Continuous)

5-element Vector{Float64}:
 185.0
 153.0
 163.0
 114.0
 180.0

Here's an example of data we would want interpreted as OrderedFactor but isn't:

exam_mark = ["rotten", "great", "bla",  missing, "great"]
scitype(exam_mark)

AbstractVector{Union{Missing, Textual}} (alias for AbstractArray{Union{Missing, ScientificTypesBase.Textual}, 1})

exam_mark = coerce(exam_mark, OrderedFactor)

5-element CategoricalArrays.CategoricalArray{Union{Missing, String},1,UInt32}:
 "rotten"
 "great"
 "bla"
 missing
 "great"

levels(exam_mark)

3-element CategoricalArrays.CategoricalArray{String,1,UInt32}:
 "bla"
 "great"
 "rotten"

Use levels! to put the classes in the right order:

levels!(exam_mark, ["rotten", "bla", "great"])
exam_mark[1] < exam_mark[2]

true

When sub-sampling, no levels are lost:

levels(exam_mark[1:2])

3-element CategoricalArrays.CategoricalArray{String,1,UInt32}:
 "rotten"
 "bla"
 "great"

Note on binary data. There is no separate scientific type for binary data. Binary data is OrderedFactor{2} or Multiclass{2}. If a binary measure like truepositive is a applied to OrderedFactor{2} then the "positive" class is assumed to appear second in the ordering. If such a measure is applied to Multiclass{2} data, a warning is issued. A single OrderedFactor can be coerced to a single Continuous variable, for models that require this, while a Multiclass variable can only be one-hot encoded.

See also Working with Categorical Data from the MLJ manual.

Two-dimensional data

Whenever it makes sense, MLJ Models generally expect two-dimensional data to be tabular. Most tabular formats implementing the Tables.jl API (see this list) have a scientific type of Table and can be used with such models.

Probably the simplest example of a table is the julia native column table, which is just a named tuple of equal-length vectors:

column_table = (h=height, e=exam_mark, t=time)

(h = [185.0, 153.0, 163.0, 114.0, 180.0], e = Union{Missing, CategoricalArrays.CategoricalValue{String, UInt32}}[CategoricalValue(CategoricalArrays.CategoricalPool{String, UInt32}(["rotten", "bla", "great"], true), 1), CategoricalValue(CategoricalArrays.CategoricalPool{String, UInt32}(["rotten", "bla", "great"], true), 3), CategoricalValue(CategoricalArrays.CategoricalPool{String, UInt32}(["rotten", "bla", "great"], true), 2), missing, CategoricalValue(CategoricalArrays.CategoricalPool{String, UInt32}(["rotten", "bla", "great"], true), 3)], t = [2.3, 4.5, 4.2, 1.8, 7.1])

While a table has a scitype, the general user will want to inspect column scitypes using MLJ's schema method:

schema(column_table)

┌───────┬──────────────────────────────────┬────────────────────────────────────
│ names │ scitypes                         │ types                             ⋯
├───────┼──────────────────────────────────┼────────────────────────────────────
│ h     │ Continuous                       │ Float64                           ⋯
│ e     │ Union{Missing, OrderedFactor{3}} │ Union{Missing, CategoricalValue{S ⋯
│ t     │ Continuous                       │ Float64                           ⋯
└───────┴──────────────────────────────────┴────────────────────────────────────
                                                                1 column omitted

Here are other examples of tables:

dict_table = Dict(:h => height, :e => exam_mark, :t => time)
schema(dict_table)

┌───────┬──────────────────────────────────┬────────────────────────────────────
│ names │ scitypes                         │ types                             ⋯
├───────┼──────────────────────────────────┼────────────────────────────────────
│ e     │ Union{Missing, OrderedFactor{3}} │ Union{Missing, CategoricalValue{S ⋯
│ h     │ Continuous                       │ Float64                           ⋯
│ t     │ Continuous                       │ Float64                           ⋯
└───────┴──────────────────────────────────┴────────────────────────────────────
                                                                1 column omitted

(To control column order here, instead use LittleDict from OrderedCollections.jl.)

row_table = [(a=1, b=3.4),
             (a=2, b=4.5),
             (a=3, b=5.6)]
schema(row_table)

┌───────┬────────────┬─────────┐
│ names │ scitypes   │ types   │
├───────┼────────────┼─────────┤
│ a     │ Count      │ Int64   │
│ b     │ Continuous │ Float64 │
└───────┴────────────┴─────────┘

import DataFrames
df = DataFrames.DataFrame(column_table)

schema(df)

┌───────┬──────────────────────────────────┬────────────────────────────────────
│ names │ scitypes                         │ types                             ⋯
├───────┼──────────────────────────────────┼────────────────────────────────────
│ h     │ Continuous                       │ Float64                           ⋯
│ e     │ Union{Missing, OrderedFactor{3}} │ Union{Missing, CategoricalValue{S ⋯
│ t     │ Continuous                       │ Float64                           ⋯
└───────┴──────────────────────────────────┴────────────────────────────────────
                                                                1 column omitted

A schema is itself a table. If we convert it to a dataframe, we can get a nicer display in some contexts (e.g., in documentation or a jupyter notebook):

schema(df) |> DataFrames.DataFrame

Most MLJ models do not accept a matrix in lieu of a table, but you can wrap a matrix as a table:

using Tables
matrix_table = Tables.table(rand(2,3))
schema(matrix_table)

┌─────────┬────────────┬─────────┐
│ names   │ scitypes   │ types   │
├─────────┼────────────┼─────────┤
│ Column1 │ Continuous │ Float64 │
│ Column2 │ Continuous │ Float64 │
│ Column3 │ Continuous │ Float64 │
└─────────┴────────────┴─────────┘

Fixing scientific types in tabular data

To show how we can correct the scientific types of data in tables, let's look more closely at a cleaned up version of the UCI Horse Colic Data set. (The cleaning work-flow is described here.)

import Downloads
import CSV
url = "https://raw.githubusercontent.com/ablaom/"*
    "MachineLearningInJulia2020/"*
    "for-MLJ-version-0.16/data/horse.csv"
csv_file = Downloads.download(url)

"/tmp/jl_fya9Vj/horse.csv"

Entering these lines of code downloads the data to a temporary file at the location shown above. We'll read this data into memory as a dataframe, provided by the DataFrames.jl package; see this tutorial for a quick-start introduction.

horse = CSV.read(csv_file, DataFrames.DataFrame)
first(horse, 4)

From the UCI docs we can surmise how each variable ought to be interpreted (a step in our work-flow that cannot reliably be left to the computer):

Let's see how MLJ will actually interpret the data, as it is currently encoded:

schema(horse)

┌─────────────────────────┬────────────┬─────────┐
│ names                   │ scitypes   │ types   │
├─────────────────────────┼────────────┼─────────┤
│ surgery                 │ Count      │ Int64   │
│ age                     │ Count      │ Int64   │
│ rectal_temperature      │ Continuous │ Float64 │
│ pulse                   │ Count      │ Int64   │
│ respiratory_rate        │ Count      │ Int64   │
│ temperature_extremities │ Count      │ Int64   │
│ mucous_membranes        │ Count      │ Int64   │
│ capillary_refill_time   │ Count      │ Int64   │
│ pain                    │ Count      │ Int64   │
│ peristalsis             │ Count      │ Int64   │
│ abdominal_distension    │ Count      │ Int64   │
│ packed_cell_volume      │ Continuous │ Float64 │
│ total_protein           │ Continuous │ Float64 │
│ outcome                 │ Count      │ Int64   │
│ surgical_lesion         │ Count      │ Int64   │
│ cp_data                 │ Count      │ Int64   │
└─────────────────────────┴────────────┴─────────┘

As a first correction step, we can get MLJ to "guess" the appropriate fix, using the autotype method:

autotype(horse)

Dict{Symbol, Type} with 11 entries:
  :abdominal_distension => OrderedFactor
  :pain => OrderedFactor
  :surgery => OrderedFactor
  :mucous_membranes => OrderedFactor
  :surgical_lesion => OrderedFactor
  :outcome => OrderedFactor
  :capillary_refill_time => OrderedFactor
  :age => OrderedFactor
  :temperature_extremities => OrderedFactor
  :peristalsis => OrderedFactor
  :cp_data => OrderedFactor

Okay, this is not perfect, but a step in the right direction, which we implement like this:

coerce!(horse, autotype(horse));
schema(horse)

┌─────────────────────────┬──────────────────┬─────────────────────────────────┐
│ names                   │ scitypes         │ types                           │
├─────────────────────────┼──────────────────┼─────────────────────────────────┤
│ surgery                 │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ age                     │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ rectal_temperature      │ Continuous       │ Float64                         │
│ pulse                   │ Count            │ Int64                           │
│ respiratory_rate        │ Count            │ Int64                           │
│ temperature_extremities │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ mucous_membranes        │ OrderedFactor{6} │ CategoricalValue{Int64, UInt32} │
│ capillary_refill_time   │ OrderedFactor{3} │ CategoricalValue{Int64, UInt32} │
│ pain                    │ OrderedFactor{5} │ CategoricalValue{Int64, UInt32} │
│ peristalsis             │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ abdominal_distension    │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ packed_cell_volume      │ Continuous       │ Float64                         │
│ total_protein           │ Continuous       │ Float64                         │
│ outcome                 │ OrderedFactor{3} │ CategoricalValue{Int64, UInt32} │
│ surgical_lesion         │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ cp_data                 │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
└─────────────────────────┴──────────────────┴─────────────────────────────────┘

All remaining Count data should be Continuous:

coerce!(horse, Count => Continuous);
schema(horse)

┌─────────────────────────┬──────────────────┬─────────────────────────────────┐
│ names                   │ scitypes         │ types                           │
├─────────────────────────┼──────────────────┼─────────────────────────────────┤
│ surgery                 │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ age                     │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ rectal_temperature      │ Continuous       │ Float64                         │
│ pulse                   │ Continuous       │ Float64                         │
│ respiratory_rate        │ Continuous       │ Float64                         │
│ temperature_extremities │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ mucous_membranes        │ OrderedFactor{6} │ CategoricalValue{Int64, UInt32} │
│ capillary_refill_time   │ OrderedFactor{3} │ CategoricalValue{Int64, UInt32} │
│ pain                    │ OrderedFactor{5} │ CategoricalValue{Int64, UInt32} │
│ peristalsis             │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ abdominal_distension    │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ packed_cell_volume      │ Continuous       │ Float64                         │
│ total_protein           │ Continuous       │ Float64                         │
│ outcome                 │ OrderedFactor{3} │ CategoricalValue{Int64, UInt32} │
│ surgical_lesion         │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ cp_data                 │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
└─────────────────────────┴──────────────────┴─────────────────────────────────┘

We'll correct the remaining truant entries manually:

coerce!(horse,
        :surgery               => Multiclass,
        :age                   => Multiclass,
        :mucous_membranes      => Multiclass,
        :capillary_refill_time => Multiclass,
        :outcome               => Multiclass,
        :cp_data               => Multiclass);
schema(horse)

┌─────────────────────────┬──────────────────┬─────────────────────────────────┐
│ names                   │ scitypes         │ types                           │
├─────────────────────────┼──────────────────┼─────────────────────────────────┤
│ surgery                 │ Multiclass{2}    │ CategoricalValue{Int64, UInt32} │
│ age                     │ Multiclass{2}    │ CategoricalValue{Int64, UInt32} │
│ rectal_temperature      │ Continuous       │ Float64                         │
│ pulse                   │ Continuous       │ Float64                         │
│ respiratory_rate        │ Continuous       │ Float64                         │
│ temperature_extremities │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ mucous_membranes        │ Multiclass{6}    │ CategoricalValue{Int64, UInt32} │
│ capillary_refill_time   │ Multiclass{3}    │ CategoricalValue{Int64, UInt32} │
│ pain                    │ OrderedFactor{5} │ CategoricalValue{Int64, UInt32} │
│ peristalsis             │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ abdominal_distension    │ OrderedFactor{4} │ CategoricalValue{Int64, UInt32} │
│ packed_cell_volume      │ Continuous       │ Float64                         │
│ total_protein           │ Continuous       │ Float64                         │
│ outcome                 │ Multiclass{3}    │ CategoricalValue{Int64, UInt32} │
│ surgical_lesion         │ OrderedFactor{2} │ CategoricalValue{Int64, UInt32} │
│ cp_data                 │ Multiclass{2}    │ CategoricalValue{Int64, UInt32} │
└─────────────────────────┴──────────────────┴─────────────────────────────────┘

Resources for this tutorial

• From the MLJ manual:
  • A preview of data type specification in MLJ
  • Data containers and scientific types
  • Working with Categorical Data
• Summary of the MLJ convention for representing scientific types
• ScientificTypes.jl
• From Data Science Tutorials:
  • Data interpretation: Scientific Types
  • Horse colic data UCI Horse Colic Data Set
• Documentation on performance metrics.

Tutorial 1 Exercises

Exercise 1

Try to guess how each code snippet below will evaluate:

scitype(42);

questions = ["who", "why", "what", "when"]
scitype(questions);

elscitype(questions);

t = (3.141, 42, "how")
scitype(t);

A = rand(2, 3)

2×3 Matrix{Float64}:
 0.275669  0.983276   0.704501
 0.814838  0.0704022  0.619187

scitype(A);

elscitype(A);

using SparseArrays
Asparse = sparse(A)

2×3 SparseArrays.SparseMatrixCSC{Float64, Int64} with 6 stored entries:
 0.275669  0.983276   0.704501
 0.814838  0.0704022  0.619187

scitype(Asparse);

C = coerce(A, Multiclass);

scitype(C);

elscitype(C);

v = [1, 2, missing, 4]
scitype(v);

elscitype(v);

scitype(v[1:2]);

Can you guess at the general behavior of scitype with respect to tuples, abstract arrays and missing values? The answers are here (ignore "Property 1").

Exercise 2

Coerce the following vector to make MLJ recognize it as a vector of ordered factors (with an appropriate ordering):

quality = ["good", "poor", "poor", "excellent", missing, "good", "excellent"];

Exercise 3 (fixing scitypes in a table)

Fix the scitypes for the House Prices in King County dataset:

url = "https://raw.githubusercontent.com/ablaom/"*
    "MachineLearningInJulia2020/for-MLJ-version-0.16/"*
    "data/house.csv";
house = CSV.read(Downloads.download(url), DataFrames.DataFrame)
first(house, 4)

(Two features in the original data set have been deemed uninformative and dropped, namely :id and :date. The original feature :yr_renovated has been replaced by the Bool feature is_renovated.)

This page was generated using Literate.jl.