Leveraging Small Language Models in NuNet Containers for Efficient ML Deployments

Estimation

Story points: 30 SP
Estimated focus duration (perfect conditions): 30 days
Estimated pessimistic duration (worst case scenario): 60 days

Description

Click to expand

Who

  1. @avimanyu786
  2. @ALISHAHMUHAMMAD - collaborator for task 4

What

  1. The integration of Small Language Models (SLMs), specifically the phi-2 model (license under the MIT License), into NuNet containers to enhance machine learning deployments with focused accuracy, minimized resources, and enhanced privacy.

How

  1. Conduct a feasibility study on SLM integration into NuNet containers, focusing on the phi-2 model.
  2. Optimize NuNet containers to support the deployment and efficient operation of SLMs.
  3. Benchmark the performance and resource utilization of SLMs within NuNet containers against traditional models.
  4. Engage with the ML/AI community for feedback on SLM integration into NuNet containers.
  5. Develop comprehensive documentation and guidelines for SLM deployment in NuNet containers.

Why

  1. To leverage the efficiency, performance, and privacy advantages of SLMs like phi-2, making AI more accessible and sustainable while enhancing NuNet's container ecosystem.

When

  1. The project kick-off is scheduled since March 26 2024.

Acceptance Criteria

Click to expand
  1. Successful deployment of the phi-2 model within a NuNet container, verified through comprehensive testing.
  2. Demonstrated efficiency and performance improvements of SLMs in NuNet containers, supported by benchmarking results.
  3. Documentation and guidelines for deploying SLMs in NuNet containers are complete and accessible.
  4. Positive feedback and engagement from the ML/AI community on the integration of SLMs into NuNet containers.
  5. All necessary optimizations for NuNet containers to support SLMs are identified and implemented.

Screenshots

Click to expand image image image

Benchmarks

Click to expand image image

Work Breakdown Structure (WBS)

Task Description Duration Status Start Date End Date Comment
1 Feasibility study for SLM integration x Hrs In Progress March 26 2024 Initial assessment and planning
2 Optimization of NuNet containers x Hrs In Progress April 11 2024 Includes performance and privacy enhancements
3 Benchmarking and testing x Hrs In Progress April 22 2024 Performance comparison and validation
4 Community engagement and feedback collection x Hrs In Progress April 29 2024 Outreach and analysis of feedback
5 Documentation and guidelines development x Hrs In Progress June 3 2024 Creation of user-friendly materials

References

  1. The Rise of Small Language Models

  2. Small Language Models Gaining Ground at Enterprises

  3. Small language models an emerging GenAI force

  4. Mini-Giants: "Small" Language Models

  5. Small language models (SLMs) simplified

  6. The Rise of Small Language Models (SLMs)

  7. Small language models emerge for domain-specific use cases

  8. Comprehensive List of Small LLMs, the Mini-Giants of the LLM World

  9. Microsoft’s 2.7 Billion Parameter Model Outperforms Llama 2, Gemini Nano 2

  10. How I Deployed a ChatGPT3 Look-alike in Less Than 5 Minutes with Python and Phi-2

Edited by Avimanyu Bandyopadhyay