Conversation
`appTags` field and helper
helper class/modules for wrapping promptable LLMs
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #292 +/- ##
==========================================
- Coverage 64.48% 63.85% -0.64%
==========================================
Files 8 9 +1
Lines 887 1126 +239
==========================================
+ Hits 572 719 +147
- Misses 315 407 +92
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
This release introduces a specialized base class
ClamsPromptableAppand its HuggingFace specializationClamsHFPromptableAppfor CLAMS apps that wrap LLM / VLM / ALM / LMM-based analyzers, along with other HF-related helper code in various places. Additionally, it addsappTagsfield for app categorization in app metadata, and new timeframe related local utility recipe forclams develop --recipe utl-tfscaffold.Additions
ClamsPromptableApp, a new specialized base class (subclassingClamsApp) for prompt-driven, instruction-tuned LLM apps with pre-equipped with commonly used runtime parameter set (prompt,systemPrompt,promptMode,maxNewTokens,temperature,topP,topK,parallelPrompts). Plus,ClamsHFPromptableApp(subclassingClamsPromptableApp) for the common case of "promptable app + local HuggingFacetransformersmodel". This subclass should significanly simplify wrapping HF models into a CLAMS app. See https://clams.ai/clams-python/app-baseclasses.html for the developer guide. (helper class/modules for wrapping promptable LLMs #291)clams.backends.hf(new module) with two general HF loaders:load_hf_model(from_pretrainedflow) andload_hf_pipeline(transformers.pipelineflow). Wraps the device auto-detection, revision pinning, dtype handling, and kwargs pass-through that every HF-backed CLAMS app does identically; used internally byClamsHFPromptableAppand callable directly from any non-promptable HF app. Requires the[hf]extra (pip install clams-python[hf]) to keeptorchandtransformersout of the base install.clams develop --recipe utl-tfscaffold drops autils/timeframe.pyinto the new app that factors out the canonical TimeFrame iteration /vdh.extract_images_by_mode_with_sources/ TimePoint-on-fallback / task-tuple pattern. (7bc91d9)appTagsfield onAppMetadata, a sibling to addedappTagsfield to view metadata mmif#253. Allows categorization of CLAMS apps for app-directory search / filtering. (appTagsfield and helper #290)Changes
mmif-pythonSDK (1.5.0).AppMetadata.analyzer_versionsfield (introduced in 1.4.0) for CHFPApp-based development. When an app subclassesClamsHFPromptableApp, the dict is now the source of truth for the family of supported HF model IDs and their pinned commit hashes: the SDK auto-derives amodelruntime parameter from the dict's keys, and the HF base class refinesmodel=<id>intomodel=<id>@<revision>on output so the resolved revision lands inview.metadata.appConfigurationautomatically.clams developscaffold templates (app.py.template,metadata.py.template) updated with commented-out blocks for both promptable variants.