Skip to content

ASHA

Local Model Advisor

AjayRajan05/privySHA

AshaFit - Local Model Advisor¶

Preview in v0.4.2 - APIs may change before 1.0.0.

Recommends local LLMs for your prompt workload and hardware.

Quick start¶

from asha.runtime.local_advisor.advisor import recommend_local_model

report = recommend_local_model(
    prompts=[
        "My email is john@company.com - analyze this dataset.",
        "Write a Python function to validate API keys.",
    ],
    mode="strict",
    top=3,
)

print(report.top_pick.model_id)
print(report.top_pick.ollama_pull_name)

CLI:

asha recommend --prompt "Analyze dataset" --gpu "RTX 4090"
asha recommend --prompts ./benchmarks/sample_prompts.json --top 3

Agent integration¶

from asha import Agent

agent = Agent(
    model="llama3",
    local_model="auto",
    sample_prompts=["Summarize this report."],
)

wrap_llm + local selection¶

from asha.integrations import wrap_llm

client = wrap_llm(
    ollama_client,
    auto_select_local_model=True,
    sample_prompts=["Your typical app prompts..."],
)

Optional dependencies¶

pip install asha[local-advisor]
pip install asha[local-advisor-gpu]   # NVIDIA VRAM detection

Scoring¶

Combines hardware fit, workload fit (from compiled prompt stats), privacy requirements, and estimated speed.