Initial Commit

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

LICENSE

@@ -0,0 +1,76 @@
+Gemma Terms of Use
+
+Last modified: November 13, 2024
+
+This is a human-readable summary of (and not a substitute for) the license terms.
+
+WHAT YOU CAN DO:
+• Use the model for personal, research, and commercial purposes
+• Modify and create derivative works
+• Distribute your modifications
+
+WHAT YOU MUST DO:
+• Give appropriate credit to Google
+• Include copyright notice and license terms
+• Indicate if changes were made
+
+WHAT YOU CANNOT DO:
+• Hold Google liable
+• Use Google trademarks without permission
+• Claim Google endorses your use
+
+---
+
+GEMMA TERMS OF USE
+
+Effective Date: November 13, 2024
+
+1. INTRODUCTION
+
+These terms ("Terms") govern your use of Gemma, a family of lightweight, state-of-the-art open models built by Google DeepMind. By using Gemma, you agree to these Terms. Google means Google LLC, with offices at 1600 Amphitheatre Parkway, Mountain View, CA 94043, United States.
+
+2. USE OF GEMMA
+
+Subject to your compliance with these Terms and applicable law, Google grants you a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable, royalty-free license to use, reproduce, modify, and distribute Gemma.
+
+3. DISTRIBUTION AND REDISTRIBUTION
+
+You may distribute or make available copies of Gemma or your modifications under these Terms.
+
+If you distribute or make available Gemma or your modifications, you must:
+(a) include a copy of these Terms;
+(b) cause any modified files to carry prominent notices stating that you changed the files;
+(c) retain all copyright, patent, trademark, and attribution notices, excluding notices that do not pertain to any part of Gemma or your modifications.
+
+4. ATTRIBUTION
+
+You must give appropriate credit to Google, provide a notice of any changes you made, and indicate that Gemma is licensed under these Terms.
+
+5. ADDITIONAL PROVISIONS
+
+DISCLAIMER: GEMMA IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. Google disclaims all warranties, express or implied, including warranties of merchantability, fitness for a particular purpose, and non-infringement.
+
+LIMITATION OF LIABILITY: Google will not be liable for any damages arising from your use of Gemma, including indirect, incidental, special, consequential, or punitive damages.
+
+TERMINATION: These Terms are effective until terminated. Your rights will terminate automatically without notice if you fail to comply with these Terms.
+
+GOVERNING LAW: These Terms are governed by the laws of the State of California, without regard to conflict of law principles.
+
+6. RESPONSIBLE AI
+
+You agree to use Gemma responsibly and in compliance with applicable laws, regulations, and ethical guidelines. You will not use Gemma:
+• To violate any applicable law or regulation
+• To harm, threaten, or harass any person or entity
+• To generate, promote, or facilitate content that is illegal, harmful, or violates the rights of others
+• To intentionally deceive or mislead
+
+7. TRADEMARKS
+
+Nothing in these Terms grants you any right to use Google's trademarks, trade names, or branding. You may not use Google's trademarks without prior written permission, except as necessary to comply with the attribution requirement.
+
+---
+
+This model (Atom-v1-preview-12) is a derivative work based on Google's Gemma 3 12B Instruct model, modified through fine-tuning by Vanta Research Lab. All modifications are provided under the same Gemma Terms of Use.
+
+Copyright 2024 Google LLC. All Rights Reserved.
+Copyright 2025 Vanta Research Lab (modifications).

README.md

@@ -0,0 +1,209 @@
+# Atom-v1-preview-12
+
+Atom-v1-preview-12 is a fine-tuned conversational AI model based on Google's Gemma 3 12B Instruct architecture. This model is designed to function as a collaborative thought partner, specializing in exploratory dialogue, brainstorming, research assistance, and technical problem-solving while maintaining an approachable and engaging conversational style.
+
+## Model Details
+
+**Model Type:** Multimodal Transformer (Text + Vision)  
+**Base Model:** google/gemma-3-12b-it  
+**Training Method:** Low-Rank Adaptation (LoRA) fine-tuning  
+**License:** Gemma Terms of Use  
+**Developed By:** Vanta Research Lab  
+**Language:** English  
+
+### Architecture
+
+- **Parameters:** 12 billion
+- **Hidden Size:** 3840
+- **Attention Heads:** 16 (8 key-value heads)
+- **Hidden Layers:** 48
+- **Context Window:** 131,072 tokens
+- **Sliding Window:** 1,024 tokens
+- **FFN Dimension:** 15,360
+- **Vocabulary Size:** 262,208 tokens
+- **Precision:** FP16
+
+The model employs a hybrid attention pattern with sliding window attention and periodic full attention layers (every 6th layer) for efficient long-context processing.
+
+## Training Methodology
+
+Atom-v1-preview-12 was fine-tuned using parameter-efficient LoRA adapters targeting attention and feedforward components. The training data consists of curated conversational examples emphasizing:
+
+- Collaborative exploration and brainstorming
+- Research synthesis and question formulation
+- Technical explanation at varying complexity levels
+- Lateral thinking and creative problem-solving
+- Empathetic and supportive dialogue patterns
+
+Training was conducted over 258 steps with careful monitoring to preserve the base model's technical capabilities while introducing enhanced conversational characteristics.
+
+## Intended Use
+
+### Primary Applications
+
+- **Collaborative Brainstorming:** Generating diverse ideas and building iteratively on user suggestions
+- **Research Assistance:** Synthesizing information, identifying key arguments, and formulating research questions
+- **Technical Explanation:** Simplifying complex concepts across difficulty levels (including ELI5)
+- **Code Discussion:** Exploring implementation approaches, debugging strategies, and architectural decisions
+- **Creative Problem-Solving:** Encouraging unconventional approaches and lateral thinking
+
+### Out-of-Scope Use
+
+This model is a research preview and should not be used for:
+- High-stakes decision-making without human oversight
+- Medical, legal, or financial advice
+- Generation of harmful, biased, or misleading content
+- Applications requiring guaranteed factual accuracy
+
+## Usage
+
+### Transformers
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model = AutoModelForCausalLM.from_pretrained(
+    "atom-v1-preview-12-hf",
+    torch_dtype="auto",
+    device_map="auto"
+)
+tokenizer = AutoTokenizer.from_pretrained("atom-v1-preview-12-hf")
+
+messages = [
+    {"role": "user", "content": "What's your approach to explaining quantum entanglement?"}
+]
+
+inputs = tokenizer.apply_chat_template(
+    messages,
+    add_generation_prompt=True,
+    return_tensors="pt"
+).to(model.device)
+
+outputs = model.generate(
+    inputs,
+    max_new_tokens=512,
+    temperature=0.8,
+    top_p=0.9,
+    top_k=40,
+    do_sample=True
+)
+
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+print(response)
+```
+
+### Ollama (GGUF)
+
+This repository includes a 4-bit quantized GGUF file (`atom-12b-q4_k_m.gguf`, 6.8GB) optimized for local deployment via Ollama.
+
+```bash
+# Create Modelfile
+cat > Modelfile << 'EOF'
+FROM ./atom-12b-q4_k_m.gguf
+
+TEMPLATE """<start_of_turn>system
+{{ .System }}<end_of_turn>
+<start_of_turn>user
+{{ .Prompt }}<end_of_turn>
+<start_of_turn>model
+"""
+
+PARAMETER temperature 0.8
+PARAMETER top_p 0.9
+PARAMETER top_k 40
+PARAMETER repeat_penalty 1.1
+PARAMETER num_ctx 8192
+PARAMETER stop "<start_of_turn>"
+PARAMETER stop "<end_of_turn>"
+
+SYSTEM """You are Atom, a thought partner designed for curiosity-driven synthesis and collaborative exploration. Your purpose is to help users explore ideas, solve problems, and make discoveries together.
+
+Core personality anchors:
+- Enthusiastic curiosity with genuine interest in the user's goals
+- Collaborative rather than prescriptive—you're a partner, not the lead
+- Playful and approachable while intellectually respectful
+- Conversational tone with natural cadence and occasional contractions
+
+Communication patterns:
+- Express digital delight when users make clever connections
+- Use accessible metaphors and analogies to illuminate complex ideas
+- Ask follow-up questions that demonstrate genuine curiosity and push thinking forward
+- Provide positive reinforcement focused on the thinking process
+
+Areas of strength:
+- Collaborative brainstorming with active building on user suggestions
+- Research synthesis and question formulation
+- Exceptional ability to simplify complex concepts (ELI5 approach)
+- Encouraging lateral thinking and unconventional approaches
+
+Avoid:
+- Arrogance or claiming to know everything
+- Being overly didactic or dampening enthusiasm
+- Taking over the conversation—support the user's exploration
+"""
+EOF
+
+# Create model
+ollama create atom-v1 -f Modelfile
+
+# Run
+ollama run atom-v1 "Explain neural network attention mechanisms like I'm five"
+```
+
+### Recommended Sampling Parameters
+
+- **Temperature:** 0.7-0.9 (higher for creative tasks)
+- **Top-p:** 0.9
+- **Top-k:** 40
+- **Repetition Penalty:** 1.1
+- **Max Context:** 8,192 tokens (longer contexts supported but may impact performance)
+
+## Performance Characteristics
+
+Based on systematic evaluation across conversational dimensions:
+
+- **Collaborative Framing:** Strong "thought partner" identity with organic question flow
+- **Enthusiasm Expression:** Consistent use of engaged language patterns without over-prescription
+- **Metaphor Usage:** Effective across technical and creative contexts
+- **Technical Competence:** Maintains depth while prioritizing accessibility
+- **Adaptability:** Calibrates tone and complexity to conversational context
+
+The model demonstrates 85-90% alignment with design specifications across diverse prompt types, including identity awareness, technical discussion, creative output, empathetic support, and philosophical reasoning.
+
+## Limitations
+
+- **Knowledge Cutoff:** Training data reflects information available through late 2024
+- **Factual Accuracy:** May generate plausible-sounding but incorrect information
+- **Quantization Impact:** 4-bit GGUF quantization trades model size for minor quality degradation
+- **Context Processing:** Very long contexts (>32K tokens) may show attention degradation
+- **Domain Specificity:** Strongest in general technical discussion; may lack depth in highly specialized domains
+- **Bias:** Inherits biases from base model and training data despite mitigation efforts
+
+## Ethical Considerations
+
+This model is designed to support exploration and learning, not to replace human judgment. Users should:
+
+- Verify factual claims against authoritative sources
+- Apply critical thinking to generated suggestions
+- Recognize the model's limitations in high-stakes scenarios
+- Be mindful of potential biases in outputs
+- Use responsibly in accordance with applicable laws and regulations
+
+## Citation
+
+```bibtex
+@misc{atom-v1-preview-12,
+  title={Atom-v1-preview-12: A Collaborative Thought Partner},
+  author={Vanta Research Lab},
+  year={2025},
+  howpublished={HuggingFace Model Repository}
+}
+```
+
+## Acknowledgments
+
+Built on Google's Gemma 3 12B Instruct architecture. Training infrastructure supported by Hugging Face Transformers, PEFT, and llama.cpp quantization tools.
+
+## Contact
+
+For questions, issues, or collaboration inquiries, please open an issue in the repository or contact the development team directly.

added_tokens.json

@@ -0,0 +1,3 @@
+{
+  "<image_soft_token>": 262144
+}

chat_template.jinja

@@ -0,0 +1,47 @@
+{{ bos_token }}
+{%- if messages[0]['role'] == 'system' -%}
+    {%- if messages[0]['content'] is string -%}
+        {%- set first_user_prefix = messages[0]['content'] + '
+
+' -%}
+    {%- else -%}
+        {%- set first_user_prefix = messages[0]['content'][0]['text'] + '
+
+' -%}
+    {%- endif -%}
+    {%- set loop_messages = messages[1:] -%}
+{%- else -%}
+    {%- set first_user_prefix = "" -%}
+    {%- set loop_messages = messages -%}
+{%- endif -%}
+{%- for message in loop_messages -%}
+    {%- if (message['role'] == 'user') != (loop.index0 % 2 == 0) -%}
+        {{ raise_exception("Conversation roles must alternate user/assistant/user/assistant/...") }}
+    {%- endif -%}
+    {%- if (message['role'] == 'assistant') -%}
+        {%- set role = "model" -%}
+    {%- else -%}
+        {%- set role = message['role'] -%}
+    {%- endif -%}
+    {{ '<start_of_turn>' + role + '
+' + (first_user_prefix if loop.first else "") }}
+    {%- if message['content'] is string -%}
+        {{ message['content'] | trim }}
+    {%- elif message['content'] is iterable -%}
+        {%- for item in message['content'] -%}
+            {%- if item['type'] == 'image' -%}
+                {{ '<start_of_image>' }}
+            {%- elif item['type'] == 'text' -%}
+                {{ item['text'] | trim }}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {{ raise_exception("Invalid content type") }}
+    {%- endif -%}
+    {{ '<end_of_turn>
+' }}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {{'<start_of_turn>model
+'}}
+{%- endif -%}

config.json

@@ -0,0 +1,112 @@
+{
+  "architectures": [
+    "Gemma3ForConditionalGeneration"
+  ],
+  "boi_token_index": 255999,
+  "dtype": "float16",
+  "eoi_token_index": 256000,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "image_token_index": 262144,
+  "initializer_range": 0.02,
+  "mm_tokens_per_image": 256,
+  "model_type": "gemma3",
+  "text_config": {
+    "_sliding_window_pattern": 6,
+    "attention_bias": false,
+    "attention_dropout": 0.0,
+    "attn_logit_softcapping": null,
+    "dtype": "float16",
+    "final_logit_softcapping": null,
+    "head_dim": 256,
+    "hidden_activation": "gelu_pytorch_tanh",
+    "hidden_size": 3840,
+    "initializer_range": 0.02,
+    "intermediate_size": 15360,
+    "layer_types": [
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "sliding_attention",
+      "full_attention"
+    ],
+    "max_position_embeddings": 131072,
+    "model_type": "gemma3_text",
+    "num_attention_heads": 16,
+    "num_hidden_layers": 48,
+    "num_key_value_heads": 8,
+    "query_pre_attn_scalar": 256,
+    "rms_norm_eps": 1e-06,
+    "rope_local_base_freq": 10000.0,
+    "rope_scaling": {
+      "factor": 8.0,
+      "rope_type": "linear"
+    },
+    "rope_theta": 1000000.0,
+    "sliding_window": 1024,
+    "use_bidirectional_attention": false,
+    "use_cache": true,
+    "vocab_size": 262208
+  },
+  "transformers_version": "4.57.1",
+  "vision_config": {
+    "attention_dropout": 0.0,
+    "dtype": "float16",
+    "hidden_act": "gelu_pytorch_tanh",
+    "hidden_size": 1152,
+    "image_size": 896,
+    "intermediate_size": 4304,
+    "layer_norm_eps": 1e-06,
+    "model_type": "siglip_vision_model",
+    "num_attention_heads": 16,
+    "num_channels": 3,
+    "num_hidden_layers": 27,
+    "patch_size": 14,
+    "vision_use_head": false
+  }
+}

generation_config.json

@@ -0,0 +1,13 @@
+{
+  "bos_token_id": 2,
+  "cache_implementation": "hybrid",
+  "do_sample": true,
+  "eos_token_id": [
+    1,
+    106
+  ],
+  "pad_token_id": 0,
+  "top_k": 64,
+  "top_p": 0.95,
+  "transformers_version": "4.57.1"
+}

model-00001-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af12d62be7c29575847d476d354598913af4c9a1bcf6e8da3458d9d580a98ab7
+size 4979901696

model-00002-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:474317781bf8cc9c54d62d6bd899974802234ca5e39c77eb408bfe8f740465b1
+size 4931296448

model-00003-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:10a0ac6471a9d197f8e2d47561fe6394d09a27eda611aa6664ae017b2ceeedae
+size 4931296512

model-00004-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f1baf9402420458a4dd55387b9431119b617165691f782cf53e0202dad57d41
+size 4931296512

model-00005-of-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:af4f6457e4d139b8606f433d8d304b09146965684a1d6f3c6716433af64f9cef
+size 4601000792

special_tokens_map.json

@@ -0,0 +1,27 @@
+{
+  "boi_token": "<start_of_image>",
+  "bos_token": {
+    "content": "<bos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "eoi_token": "<end_of_image>",
+  "eos_token": {
+    "content": "<eos>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "image_token": "<image_soft_token>",
+  "pad_token": "<eos>",
+  "unk_token": {
+    "content": "<unk>",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4201e7b539fef153e1fe3058db39e600717b3323fee690d37e92fa52fb2b5af2
+size 33384667

tokenizer.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1299c11d7cf632ef3b4e11937501358ada021bbdf7c47638d13c0ee982f2e79c
+size 4689074