Add test for scoring a pipeline with Runtime

  • We should add the scoring pipeline into core package, probably store it under d3m.contrib.pipelines.
  • Make sure that when a d3m package is made, pipeline is included. Try with python3 setup.py sdist and check the contents of the file.
  • You should probably name the pipeline based on its IDS.
  • You should move compute scores primitive to core package, probably under d3m.contrib.primitives.
  • It should be slightly different from regular primitives: no need for installation section and source sections in metadata.
  • Make sure it is registered: try to add it to entry points of core package, see if that works.
  • You should probably rename it to d3m.primitives.evaluation.compute_scores.Core or something.
  • In runtime command line, make it so that the scoring pipeline above is the default is some other scoring pipeline is not provided (for arguments which are to define the scoring pipeline to use).
  • Add top-level command line tests for fit-score which uses the default scoring pipeline.
Edited by linyang