Training update: 110/67,618 rows (0.16%) | +50 new @ 2025-10-20 05:30:24

README.md

@@ -1,54 +1,70 @@
 ---
-library_name: transformers
-license: mit
-base_model: boltuix/bert-micro
+language:
+- en
+- id
 tags:
-- generated_from_trainer
-model-index:
-- name: bert-micro-cybersecurity
-  results: []
+- text-classification
+- cybersecurity
+base_model: boltuix/bert-micro
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 # bert-micro-cybersecurity
 
-This model is a fine-tuned version of [boltuix/bert-micro](https://huggingface.co/boltuix/bert-micro) on the None dataset.
+## 1. Model Details
+
+**Model description**  
+"bert-micro-cybersecurity" is a compact transformer model derived from `boltuix/bert-micro`, adapted for cybersecurity text classification tasks (e.g., threat detection, incident reports, malicious vs benign content).
+
+- Model type: fine-tuned lightweight BERT variant  
+- Languages: English & Indonesia
+- Finetuned from: `boltuix/bert-micro`
+- Status: **Early version** — trained on **0.16%** of planned data).
 
-## Model description
+**Model sources**  
+- Base model: [boltuix/bert-micro](https://huggingface.co/boltuix/bert-micro)
+- Data: Cybersecurity Data
 
-More information needed
+## 2. Uses
 
-## Intended uses & limitations
+### Direct use  
+You can use this model to classify cybersecurity-related text — for example, whether a given message, report or log entry indicates malicious intent, abnormal behaviour, or threat presence.
 
-More information needed
+### Downstream use  
+- Embedding extraction for clustering or anomaly detection in security logs.  
+- As part of a pipeline for phishing detection, malicious email filtering, incident triage.  
+- As a feature extractor feeding a downstream system (e.g., alert-generation, SOC dashboard).
 
-## Training and evaluation data
+### Out-of-scope use  
+- Not meant for high-stakes automated blocking decisions without human review.  
+- Not optimized for languages other than English and Indonesian.  
+- Not tested for non-cybersecurity domains or out-of-distribution data.
 
-More information needed
+## 3. Bias, Risks, and Limitations 
 
-## Training procedure
+Because the model is based on a small subset (0.16%) of planned data, performance is preliminary and may degrade on unseen or specialized domains (industrial control, IoT logs, foreign language).
 
-### Training hyperparameters
+- Inherits any biases present in the base model (`boltuix/bert-micro`) and in the fine-tuning data — e.g., over-representation of certain threat types, vendor or tooling-specific vocabulary.
+- Should not be used as sole authority for incident decisions; only as an aid to human analysts.
 
-The following hyperparameters were used during training:
-- learning_rate: 5e-05
-- train_batch_size: 8
-- eval_batch_size: 8
-- seed: 42
-- optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.06
-- num_epochs: 3
+## 4. How to Get Started with the Model  
 
-### Training results
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
 
+tokenizer = AutoTokenizer.from_pretrained("codechrl/bert-micro-cybersecurity")
+model = AutoModelForSequenceClassification.from_pretrained("codechrl/bert-micro-cybersecurity")
 
+inputs = tokenizer("The server logged an unusual outbound connection to 123.123.123.123", 
+                   return_tensors="pt", truncation=True, padding=True)
+outputs = model(**inputs)
+logits = outputs.logits
+predicted_class = logits.argmax(dim=-1).item()
+```
 
-### Framework versions
+## 5. Training Details
 
-- Transformers 4.57.0
-- Pytorch 2.8.0+cu128
-- Datasets 4.2.0
-- Tokenizers 0.22.1
+- **Trained records**: 110 / 67,618 (0.16%)
+- **Learning rate**: 5e-05
+- **Epochs**: 3
+- **Batch size**: 8
+- **Max sequence length**: 512

checkpoint-21/config.json

@@ -0,0 +1,24 @@
+{
+  "architectures": [
+    "BertForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30522
+}

checkpoint-21/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fccffc8436212a980c3a25ef5c4583dfed723e4f0c1c8857ea354d783347165c
+size 17671560

checkpoint-21/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8e0a700261a075b2798ae3c3216f142d1949a2c849645138e8de3eb2613a50d6
+size 35368075

checkpoint-21/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e61e0305d9611ca4c75692d1a176f35005d0ec603a42601986f9763caf8eb903
+size 14455

checkpoint-21/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f374c8187c2d2db1143eb348871ee957c3603763a041fd251d09062f56d5688
+size 1465

checkpoint-21/special_tokens_map.json

@@ -0,0 +1,37 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": {
+    "content": "[PAD]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

checkpoint-21/tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 1000000000000000019884624838656,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

checkpoint-21/trainer_state.json

@@ -0,0 +1,33 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 21,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [],
+  "logging_steps": 100,
+  "max_steps": 21,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 204636672000.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-21/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dc19d58cd6f4e9fa4667b169e0aa27f6ade93802615c12a15ae7cfdf97b1d049
+size 5841

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:620ecf924382af049bf9177f69f38d213c9ecfad19d99cc31432f1303545db4b
+oid sha256:fccffc8436212a980c3a25ef5c4583dfed723e4f0c1c8857ea354d783347165c
 size 17671560

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7fbf9c2924957ebfb9c1cdf0e7b5f2c65ce144192d10d1f832db128968032ad8
+oid sha256:dc19d58cd6f4e9fa4667b169e0aa27f6ade93802615c12a15ae7cfdf97b1d049
 size 5841

training_metadata.json

@@ -1,10 +1,11 @@
 {
-  "trained_at": 1760937826.247165,
-  "trained_at_readable": "2025-10-20 05:23:46",
-  "samples": 50,
-  "trained_rows": 60,
+  "trained_at": 1760938224.2996256,
+  "trained_at_readable": "2025-10-20 05:30:24",
+  "samples_this_session": 50,
+  "new_rows_this_session": 50,
+  "trained_rows_total": 110,
   "total_db_rows": 67618,
-  "percentage": 0.08873376911473276,
+  "percentage": 0.1626785767103434,
   "final_loss": 0,
   "epochs": 3,
   "learning_rate": 5e-05