Humanloop

Reading on prompts

• Prompt, or fine-tune?Almost every 'the model does not know this' problem is a retrieval problem wearing a hat. Here is how to tell the difference before you spend a week on a training run.
• Context is a budget, not a containerThe instinct is to keep everything in case it is needed. The models are not filing systems, and past a certain length the middle of your context is doing nothing but costing money.

• PromptLayerPrompt registry with request logging and regression testing.
• AgentaOpen-source prompt management with a playground and versioning.

Listed as an alternative to

• PromptLayerPromptLayerPrompt registry with request logging and regression testing.Use when A prompt registry with request logging and regression tests.Skip when You need self-hosting.proprietary · saas · usage-based
• AgentaAgentaOpen-source prompt management with a playground and versioning.Use when Open-source prompt management with a playground and versioning.Skip when You need eval depth rather than prompt ergonomics.Apache-2.0 · self-hosted · free-tier
• PromptwatchPromptwatchPrompt regression tests and side-by-side comparison.Use when Prompt regression tests and side-by-side comparison.Skip when You need self-hosting.proprietary · saas · usage-based
• VellumVellumPrompt and eval platform with a visual debugger for chains.Use when Prompt and eval work with a visual debugger for chains.Skip when Self-hosting or reproducibility is a requirement.proprietary · saas · usage-based

Also in prompts

• DSPyDSPyDeclarative prompting that compiles to optimized programs from examples.Use when Compiling prompts into optimised programs from labelled examples.Skip when You have no examples and no way to score them.MIT · self-hosted · free
• InstructorInstructorSchema-constrained extraction with automatic validation and retry.Use when Schema-validated extraction with typed retry and partial streaming.Skip when You need a hard token-level guarantee rather than validation.MIT · self-hosted · free
• GuidanceGuidanceConstrained generation by interleaving control flow and model output.Use when Constrained decoding with control flow interleaved into generation.Skip when You need better ergonomics for plain extraction.MIT · self-hosted · free
• PromptLayerPromptLayerPrompt registry with request logging and regression testing.Use when A prompt registry with request logging and regression tests.Skip when You need self-hosting.proprietary · saas · usage-based
• AgentaAgentaOpen-source prompt management with a playground and versioning.Use when Open-source prompt management with a playground and versioning.Skip when You need eval depth rather than prompt ergonomics.Apache-2.0 · self-hosted · free-tier
• PromptwatchPromptwatchPrompt regression tests and side-by-side comparison.Use when Prompt regression tests and side-by-side comparison.Skip when You need self-hosting.proprietary · saas · usage-based
• TextGradTextGradAutomatic prompt optimisation via textual feedback loops.Use when Automatic prompt optimisation via textual feedback loops.Skip when Optimising without an eval set is guessing.MIT · self-hosted · free
• OutlinesOutlinesConstrained generation that masks the token space to valid output.Use when Token-level constrained generation from a JSON schema or grammar.Skip when Post-hoc validation is enough for your use case.Apache-2.0 · self-hosted · free