Scipediacontent: Scipediacontent moved page Draft Content 847655201 to Macdonald 2018a

2021-02-02T04:50:03Z

Scipediacontent moved page Draft Content 847655201 to Macdonald 2018a

Scipediacontent: Created page with " == Abstract == Experimentation using IR systems has traditionally been a procedural and laborious process. Queries must be run on an index, with any parameters of the retrie..."

2021-02-02T04:49:57Z

Created page with " == Abstract == Experimentation using IR systems has traditionally been a procedural and laborious process. Queries must be run on an index, with any parameters of the retrie..."

New page

== Abstract ==

Experimentation using IR systems has traditionally been a procedural and laborious process. Queries must be run on an index, with any parameters of the retrieval models suitably tuned. With the advent of learning-to-rank, such experimental processes (including the appropriate folding of queries to achieve cross-fold validation) have resulted in complicated experimental designs and hence scripting. At the same time, machine learning platforms such as Scikit Learn and Apache Spark have pioneered the notion of an experimental pipeline , which naturally allows a supervised classification experiment to be expressed a series of stages, which can be learned or transformed. In this demonstration, we detail Terrier-Spark, a recent adaptation to the Terrier Information Retrieval platform which permits it to be used within the experimental pipelines of Spark. We argue that this (1) provides an agile experimental platform for information retrieval, comparable to that enjoyed by other branches of data science; (2) aids research reproducibility in information retrieval by facilitating easily-distributable notebooks containing conducted experiments; and (3) facilitates the teaching of information retrieval experiments in educational environments.

== Original document ==

The different versions of the original document can be found in:

* [http://eprints.gla.ac.uk/160904/7/160904.pdf http://eprints.gla.ac.uk/160904/7/160904.pdf]

* [http://dl.acm.org/ft_gateway.cfm?id=3210174&ftid=1982862&dwn=1 http://dl.acm.org/ft_gateway.cfm?id=3210174&ftid=1982862&dwn=1],
: [http://dx.doi.org/10.1145/3209978.3210174 http://dx.doi.org/10.1145/3209978.3210174] under the license http://www.acm.org/publications/policies/copyright_policy#Background

* [https://dblp.uni-trier.de/db/conf/sigir/sigir2018.html#Macdonald18 https://dblp.uni-trier.de/db/conf/sigir/sigir2018.html#Macdonald18],
: [http://eprints.gla.ac.uk/160904 http://eprints.gla.ac.uk/160904],
: [https://dl.acm.org/citation.cfm?id=3210174 https://dl.acm.org/citation.cfm?id=3210174],
: [https://academic.microsoft.com/#/detail/2798639660 https://academic.microsoft.com/#/detail/2798639660]

← Older revision	Revision as of 04:50, 2 February 2021
(No difference)

Macdonald 2018a - Revision history

Scipediacontent: Scipediacontent moved page Draft Content 847655201 to Macdonald 2018a

Scipediacontent: Created page with " == Abstract == Experimentation using IR systems has traditionally been a procedural and laborious process. Queries must be run on an index, with any parameters of the retrie..."