Training update: 1/237,628 rows (0.00%) | +1 new @ 2025-10-20 06:45:25

README.md

@@ -0,0 +1,70 @@
+---
+language:
+- en
+- id
+tags:
+- text-classification
+- cybersecurity
+base_model: google-bert/bert-base-cased
+---
+
+# bert-base-cybersecurity
+
+## 1. Model Details
+
+**Model description**  
+"bert-base-cybersecurity" is a transformer model adapted for cybersecurity text classification tasks (e.g., threat detection, incident reports, malicious vs benign content).
+
+- Model type: fine-tuned lightweight BERT variant  
+- Languages: English & Indonesia
+- Finetuned from: `bert-base-cased`
+- Status: **Early version** — trained on **0.00%** of planned data.
+
+**Model sources**  
+- Base model: [google-bert/bert-base-cased](https://huggingface.co/google-bert/bert-base-cased)
+- Data: Cybersecurity Data
+
+## 2. Uses
+
+### Direct use  
+You can use this model to classify cybersecurity-related text — for example, whether a given message, report or log entry indicates malicious intent, abnormal behaviour, or threat presence.
+
+### Downstream use  
+- Embedding extraction for clustering or anomaly detection in security logs.  
+- As part of a pipeline for phishing detection, malicious email filtering, incident triage.  
+- As a feature extractor feeding a downstream system (e.g., alert-generation, SOC dashboard).
+
+### Out-of-scope use  
+- Not meant for high-stakes automated blocking decisions without human review.  
+- Not optimized for languages other than English and Indonesian.  
+- Not tested for non-cybersecurity domains or out-of-distribution data.
+
+## 3. Bias, Risks, and Limitations 
+
+Because the model is based on a small subset (0.00%) of planned data, performance is preliminary and may degrade on unseen or specialized domains (industrial control, IoT logs, foreign language).
+
+- Inherits any biases present in the base model (`google-bert/bert-base-cased`) and in the fine-tuning data — e.g., over-representation of certain threat types, vendor or tooling-specific vocabulary.
+- Should not be used as sole authority for incident decisions; only as an aid to human analysts.
+
+## 4. How to Get Started with the Model  
+
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+
+tokenizer = AutoTokenizer.from_pretrained("codechrl/bert-base-cybersecurity")
+model = AutoModelForSequenceClassification.from_pretrained("codechrl/bert-base-cybersecurity")
+
+inputs = tokenizer("The server logged an unusual outbound connection to 123.123.123.123", 
+                   return_tensors="pt", truncation=True, padding=True)
+outputs = model(**inputs)
+logits = outputs.logits
+predicted_class = logits.argmax(dim=-1).item()
+```
+
+## 5. Training Details
+
+- **Trained records**: 1 / 237,628 (0.00%)
+- **Learning rate**: 5e-05
+- **Epochs**: 3
+- **Batch size**: 1
+- **Max sequence length**: 512

checkpoint-3/config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "BertForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 28996
+}

checkpoint-3/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cf5af4f8435c00c5c01ecd73f491fc7c728eb4a69ac2d05e510218a3d2d14a4
+size 433386912

checkpoint-3/optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65d51d6a8cc86ee1cc8a9067b99263cb7b6d4cabb74ab12b1554587a9f9ffdf9
+size 866890187

checkpoint-3/rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8de7534ff74b964ae2187134d92ae1e13f86e8f913a9590aebf0882a2a0508e2
+size 14455

checkpoint-3/scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a1883b05d1aea2d7680e95c8580838f2a05b0260e5fc08597f62232cb8732629
+size 1465

checkpoint-3/special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

checkpoint-3/tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": false,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

checkpoint-3/trainer_state.json

@@ -0,0 +1,33 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 3.0,
+  "eval_steps": 500,
+  "global_step": 3,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [],
+  "logging_steps": 100,
+  "max_steps": 3,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 3,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": true
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 789600374784.0,
+  "train_batch_size": 1,
+  "trial_name": null,
+  "trial_params": null
+}

checkpoint-3/training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c11a953943d18125021ec6b039671808647923abd57ac008164b1365ae5cfcec
+size 5841

config.json

@@ -0,0 +1,25 @@
+{
+  "architectures": [
+    "BertForMaskedLM"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "gradient_checkpointing": false,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "transformers_version": "4.57.0",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 28996
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7cf5af4f8435c00c5c01ecd73f491fc7c728eb4a69ac2d05e510218a3d2d14a4
+size 433386912

special_tokens_map.json

@@ -0,0 +1,7 @@
+{
+  "cls_token": "[CLS]",
+  "mask_token": "[MASK]",
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "unk_token": "[UNK]"
+}

tokenizer_config.json

@@ -0,0 +1,58 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    }
+  },
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "[CLS]",
+  "do_basic_tokenize": true,
+  "do_lower_case": false,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "never_split": null,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "unk_token": "[UNK]"
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c11a953943d18125021ec6b039671808647923abd57ac008164b1365ae5cfcec
+size 5841

training_metadata.json

@@ -0,0 +1,12 @@
+{
+  "trained_at": 1760942725.2914977,
+  "trained_at_readable": "2025-10-20 06:45:25",
+  "samples_this_session": 1,
+  "new_rows_this_session": 1,
+  "trained_rows_total": 1,
+  "total_db_rows": 237628,
+  "percentage": 0.0004208258286060565,
+  "final_loss": 0,
+  "epochs": 3,
+  "learning_rate": 5e-05
+}