Hugging Face's logo
Activity Feed

AI & ML interests

None defined yet.
alvarobarttĀ 
posted an update 8 days ago
view post
Post
257
Open agents on AWS SageMaker AI with open models from the Hugging Face Hub!

> Deploy an open model from the Hugging Face Hub on SageMaker AI
> Connect the deployed model to Strands Agents
> Add built-in and custom tools for tool calling
> Expose external capabilities through MCP integration
> Bonus: talk to your agent and visualize traces with Gradio

https://alvarobartt.com/agents-on-aws-sagemaker
alvarobarttĀ 
posted an update 12 days ago
view post
Post
3261
Latest hf-mem release added a breakdown of Mixture-of-Experts (MoE) memory usage!

TL; DR MoEs can be misleading to reason about from active parameters alone, since each token only activates a subset of experts, while the serving setup still needs to account for the full resident memory footprint.

šŸ§  hf-mem now splits MoE memory into base model weights, routed experts, and KV cache
šŸ—ļø Dense models usually load and use most weights every forward pass, while MoEs load many experts but only route each token to a few of them
āš” Active params isn't the same as memory footprint, especially for sparse architectures
šŸ“¦ Runtime memory is about what is used per request/token, while loading memory also includes the expert weights that need to be resident
šŸ“š KV cache can still dominate depending on context length, batch size, and concurrency
šŸ”€ Expert Parallelism (EP) helps shard experts across accelerators when expert weights dominate
šŸš€ Data Parallelism (DP) + EP is often a good fit for throughput-oriented MoE serving

Check the repository at https://github.com/alvarobartt/hf-mem
alvarobarttĀ 
posted an update 3 months ago
view post
Post
3742
Learn how to deploy Microsoft Research VibeVoice ASR on Microsoft Azure Foundry with Hugging Face to generate rich audio transcriptions with Who, When, and What! šŸ’„

> šŸ•’ 60-minute single-pass processing, no chunking or stitching
> šŸ‘¤ Customized hotwords to guide recognition on domain-specific content
> šŸ“ Rich transcription: joint ASR + diarization + timestamping in one pass
> šŸŒ 50+ languages with automatic detection and code-switching support
> šŸ¤— Deployed on Microsoft Foundry via an OpenAI-compatible Chat Completions API

https://huggingface.co/docs/microsoft-azure/foundry/examples/deploy-vibevoice-asr
alvarobarttĀ 
posted an update 4 months ago
view post
Post
3269
šŸ’„ hf-mem v0.4.1 now also estimates KV cache memory requirements for any context length and batch size with the --experimental flag!

uvx hf-mem --model-id ... --experimental will automatically pull the required information from the Hugging Face Hub to include the KV cache estimation, when applicable.

šŸ’” Alternatively, you can also set the --max-model-len, --batch-size and --kv-cache-dtype arguments (Ć  la vLLM) manually if preferred.
  • 1 reply
Ā·
jw2yangĀ 
updated a dataset about 1 year ago
jw2yangĀ 
published a dataset about 1 year ago
jw2yangĀ 
updated a dataset about 1 year ago
jw2yangĀ 
published a dataset about 1 year ago
jw2yangĀ 
updated a dataset about 1 year ago
jw2yangĀ 
published a dataset about 1 year ago
jw2yangĀ 
updated a dataset about 1 year ago
jw2yangĀ 
published 2 datasets about 1 year ago
jw2yangĀ 
updated a dataset about 1 year ago
alvarobarttĀ 
posted an update over 1 year ago
view post
Post
3647
šŸ”„ Agents can do anything! @microsoft Research just announced the release of Magma 8B!

Magma is a new Visual Language Model (VLM) with 8B parameters for multi-modal agents designed to handle complex interactions across virtual and real environments; and it's MIT licensed!

Magma comes with exciting new features such as:
- Introduces the Set-of-Mark and Trace-of-Mark techniques for fine-tuning
- Leverages a large amount of unlabeled video data to learn the spatial-temporal grounding and planning
- A strong generalization and ability to be fine-tuned for other agentic tasks
- SOTA in different multi-modal benchmarks spanning across UI navigation, robotics manipulation, image / video understanding and spatial understanding and reasoning
- Generates goal-driven visual plans and actions for agentic use cases

Model: microsoft/Magma-8B
Technical Report: Magma: A Foundation Model for Multimodal AI Agents (2502.13130)
jw2yangĀ 
authored 5 papers over 1 year ago