Sathya77/sd2.1-base-Diorma-style-LoRA
Text-to-Image • Updated • 16
Santosh Kompella PRO
Sathya77
AI & ML interests
LLMs
Natural Language Processing (NLP)
Transformers
Deep Learning
Machine Learning
Recent Activity
updated a model 1 day ago
Sathya77/sd2.1-base-Diorma-style-LoRA published a model 1 day ago
Sathya77/sd2.1-base-Diorma-style-LoRA updated a Space 2 days ago
Sathya77/Miniature-Diorama-Style-Image-GeneratorOrganizations
None yet
updated a
model 1 day ago
published a
model 1 day ago
updated a
Space 2 days ago
published a
Space 3 days ago
posted an update 27 days ago
Post
120
My previous ViT-SR model handled broad structure decently well but lost fine detail. Scaling parameters alone didn't fix it — a larger model with the same spatial compression through patch merging scored worse than the baseline. The problem wasn't capacity. It was architecture.
The reconstruction head was being asked to recover spatial detail from a heavily compressed token grid — information that was already gone before upsampling began. Fixing that required a different design, not just more parameters.
What changed:
Constant 16×16 token grid across all 4 transformer stages. No patch merging, no token downsampling. Every token maps to the same 4×4 pixel region from input through to reconstruction (DenseNet's Principle to Transformers)
Hierarchical embed dims [192 → 256 → 288 → 384]. These architectural changes and the parameter count are not separable — GDFN, dense concatenation, and hierarchical dims each require sufficient channel depth to function. A 786K version of this architecture would be a fundamentally different, weaker model.
Dense concatenation of all 4 stage outputs directly to the reconstruction head — simultaneously accessing low-level edge maps and high-level semantic context.
Gated Depthwise Feed-Forward Network (GDFN) — replaces standard MLPs with gated depthwise 3×3 convolutions, injecting local spatial context into every attention step.
Bilinear skip connection — model learns residual correction only, not full reconstruction from scratch.
Results:
DIV2K validation: 25.20 dB PSNR, SSIM 0.8298
~23.8M parameters, trained from scratch on LSDIR (84,991 images)
The model handles smooth regions and structured content well. High-frequency fine detail remains the harder case — likely a property of global attention at this token count rather than anything specific to this architecture.
🤗 Space: Sathya77/Dense-Iso-ViT-SR
Feedback Welcome!
The reconstruction head was being asked to recover spatial detail from a heavily compressed token grid — information that was already gone before upsampling began. Fixing that required a different design, not just more parameters.
What changed:
Constant 16×16 token grid across all 4 transformer stages. No patch merging, no token downsampling. Every token maps to the same 4×4 pixel region from input through to reconstruction (DenseNet's Principle to Transformers)
Hierarchical embed dims [192 → 256 → 288 → 384]. These architectural changes and the parameter count are not separable — GDFN, dense concatenation, and hierarchical dims each require sufficient channel depth to function. A 786K version of this architecture would be a fundamentally different, weaker model.
Dense concatenation of all 4 stage outputs directly to the reconstruction head — simultaneously accessing low-level edge maps and high-level semantic context.
Gated Depthwise Feed-Forward Network (GDFN) — replaces standard MLPs with gated depthwise 3×3 convolutions, injecting local spatial context into every attention step.
Bilinear skip connection — model learns residual correction only, not full reconstruction from scratch.
Results:
DIV2K validation: 25.20 dB PSNR, SSIM 0.8298
~23.8M parameters, trained from scratch on LSDIR (84,991 images)
The model handles smooth regions and structured content well. High-frequency fine detail remains the harder case — likely a property of global attention at this token count rather than anything specific to this architecture.
🤗 Space: Sathya77/Dense-Iso-ViT-SR
Feedback Welcome!
updated a
Space 27 days ago
published a
Space 27 days ago
posted an update about 1 month ago
Post
594
Built a ViT for ×4 image super-resolution from scratch in PyTorch — sharing the model.
No pretrained weights. Every component implemented from scratch: strided Conv2d patch embedding, multi-head self-attention across 1,024 tokens, 6 pre-norm transformer blocks, and a PixelShuffle reconstruction head for learned upsampling.
Trained on real-images from LSDIR dataset with fp16 AMP on a laptop GPU. Tiled inference handles arbitrary input sizes.
Current architecture: patch size 2, embed dim 64, 4 attention heads, 6 transformer blocks, ~786K parameters — test PSNR 23.30 dB.
The model handles broad structure well — fine textures and sharp edges need more capacity. Working on a larger configuration next.
🤗 Space: Sathya77/ViT-ISR-Tiny-LSDIR
Feedback welcome — especially on the architecture choices.
No pretrained weights. Every component implemented from scratch: strided Conv2d patch embedding, multi-head self-attention across 1,024 tokens, 6 pre-norm transformer blocks, and a PixelShuffle reconstruction head for learned upsampling.
Trained on real-images from LSDIR dataset with fp16 AMP on a laptop GPU. Tiled inference handles arbitrary input sizes.
Current architecture: patch size 2, embed dim 64, 4 attention heads, 6 transformer blocks, ~786K parameters — test PSNR 23.30 dB.
The model handles broad structure well — fine textures and sharp edges need more capacity. Working on a larger configuration next.
🤗 Space: Sathya77/ViT-ISR-Tiny-LSDIR
Feedback welcome — especially on the architecture choices.
updated a
model about 1 month ago
published a
model about 1 month ago
updated a
Space about 1 month ago
published a
Space about 1 month ago
upvoted an article about 2 months ago
Article
Deploying Open Source Vision Language Models (VLM) on Jetson
nvidia
• • 37 liked 2
Spaces about 2 months ago
posted an update 2 months ago
Post
190
Trained a Swin-T from scratch on NWPU-RESISC45 — no pretrained weights, no fine-tuning.
Every component hand-coded in PyTorch: window partitioning, shifted window attention with relative positional bias, patch merging across 4 stages, ~28M parameters.
Architecture:
embed_dim=96, window_size=7, depths=[2, 2, 6, 2]
heads=[3, 6, 12, 24] across stages
Patch embed via Conv2d (4×4, stride 4) → 56×56 feature map
PatchMerging downsamples by concatenating 2×2 neighbors + linear projection
Global average pooling → linear classifier
Training:
AdamW (lr=3e-4, weight_decay=0.05)
Cosine annealing with 3-epoch linear warmup over 20 epochs
Mixed precision (autocast + GradScaler)
Gradient clipping (max_norm=1.0)
Label smoothing (0.1)
ImageNet normalization, batch size 32
80/20 train/test split, seed=42
Result: 82% test accuracy on 45 land-use categories, 31,500 images.
🔗 Sathya77/swin-transformer-satellite
What accuracy do you think is achievable on NWPU-RESISC45 with Swin-T trained from scratch, without any pretraining?
Every component hand-coded in PyTorch: window partitioning, shifted window attention with relative positional bias, patch merging across 4 stages, ~28M parameters.
Architecture:
embed_dim=96, window_size=7, depths=[2, 2, 6, 2]
heads=[3, 6, 12, 24] across stages
Patch embed via Conv2d (4×4, stride 4) → 56×56 feature map
PatchMerging downsamples by concatenating 2×2 neighbors + linear projection
Global average pooling → linear classifier
Training:
AdamW (lr=3e-4, weight_decay=0.05)
Cosine annealing with 3-epoch linear warmup over 20 epochs
Mixed precision (autocast + GradScaler)
Gradient clipping (max_norm=1.0)
Label smoothing (0.1)
ImageNet normalization, batch size 32
80/20 train/test split, seed=42
Result: 82% test accuracy on 45 land-use categories, 31,500 images.
🔗 Sathya77/swin-transformer-satellite
What accuracy do you think is achievable on NWPU-RESISC45 with Swin-T trained from scratch, without any pretraining?
liked a
Space 2 months ago
updated a
Space 2 months ago
updated a
model 2 months ago
published a
Space 2 months ago