Skip to content

WorkAcademic

Question–answer text classification

BRAC University · 2025

An NLP study comparing classical machine learning against recurrent architectures on a corpus of 339,998 question–answer pairs, across custom-trained and pre-trained embeddings. It was designed as a comparison rather than a demonstration, which meant the boring models had to be built first and taken seriously.

PythonTensorFlowScikit-learnNLTKGloVeWord2Vec
01

The problem

Text classification papers have a habit of reporting a deep model's score with nothing meaningful to compare it against, which makes the number impossible to interpret. A recurrent network that beats nothing has not been shown to be worth its training time, its inference cost, or its opacity. The question here was narrower and more useful: on this corpus, how much does each additional increment of model complexity actually buy?

02

The system

  1. Corpus

    339,998 pairs

  2. Preprocessing

    NLTK, identical for all

  3. Embeddings

    in-corpus, GloVe, W2V

  4. Models

    2 classical, 3 recurrent

  5. Comparison

    cost against gain

One pipeline, held fixed, with the comparison happening at the end of it. Five models over three embedding strategies is fifteen runs, and they are only comparable because nothing upstream of them varies.

A corpus of 339,998 pairs

Preprocessing through NLTK (tokenisation, normalisation, and vocabulary construction) applied identically across every model, so that differences in the results come from the models rather than from the pipeline.

Classical baselines

Logistic regression and random forest over engineered text features, built first and tuned properly rather than treated as a formality.

Recurrent architectures

LSTM, GRU, and BiLSTM in TensorFlow. The three are a deliberate progression: gated memory, a cheaper gating scheme, and then bidirectional context, so each step isolates one variable.

Three embedding strategies

Embeddings learned from the corpus itself, and pre-trained GloVe and Word2Vec vectors. The same architectures run over each, so the embedding choice can be evaluated separately from the architecture choice.

03

Technical decisions

What was chosen, what was chosen against, and what the trade cost.

01
Chose
Building and tuning classical baselines before any neural model.
Instead of
Going straight to the recurrent architectures.
Why
A neural result only means something relative to a strong simple model. If logistic regression lands within a point or two, the LSTM has not earned its training cost or its loss of interpretability, and that is a finding, not a failure. Skipping the baseline makes the headline number unfalsifiable.
02
Chose
Comparing pre-trained embeddings against embeddings learned in-corpus.
Instead of
Defaulting to pre-trained vectors, as most implementations do.
Why
340,000 pairs is an awkward size: large enough that in-corpus embeddings can learn genuine domain vocabulary, small enough that they may not beat vectors trained on billions of tokens. Which way it falls is an empirical question about this corpus, so it had to be run rather than assumed.
03
Chose
GRU and BiLSTM alongside LSTM rather than instead of it.
Instead of
Picking the architecture reported best in the literature.
Why
GRU isolates the cost of the LSTM's extra gate; BiLSTM isolates the value of reading the sequence in both directions. Run together the three answer a question the single best-performing model cannot: which specific property of the architecture the task actually rewards.
04

The result

A comparison across the full grid of architectures and embedding strategies on a single consistently preprocessed corpus, enough to say not just which model scored highest, but what each increment of complexity bought. The habit it built is the one that transferred: establish the cheap baseline first, so the expensive answer has something to be measured against.