Update README.md

README.md

@@ -1,3 +1,1933 @@
 ---
+language:
+- en
 license: mit
+tags:
+- text-generation-inference
+- transformers
+- unsloth
+- mistral
+base_model: alpindale/Mistral-7B-v0.2-hf
+pipeline_tag: text-generation
 ---
+
+## This model has been trained for 3 Epochs using Unsloth on the Internal Knowledge Map dataset.
+
+Since this is a base model the IKM dataset greatly affects the output. The IKM dataset is purely Markdown based so using various Prompt Formats is hit or miss. Mistral Instruct, Chat ML and Alpaca are ok. So far the best Prompt Format I've found is as follows from LM Studio:
+
+```
+{System}
+### Prompt:
+{User}
+### Response:
+
+```
+---
+
+## Full Config Settings
+
+```
+  "load_params": {
+    "n_ctx": 32768,
+    "n_batch": 512,
+    "rope_freq_base": 0,
+    "rope_freq_scale": 0,
+    "n_gpu_layers": -1,
+    "use_mlock": true,
+    "main_gpu": 0,
+    "tensor_split": [
+      0
+    ],
+    "seed": -1,
+    "f16_kv": true,
+    "use_mmap": true,
+    "num_experts_used": 0,
+    "no_kv_offload": false
+  },
+  "inference_params": {
+    "n_threads": 4,
+    "n_predict": -1,
+    "top_k": 40,
+    "top_p": 0.95,
+    "temp": 0.8,
+    "repeat_penalty": 1.1,
+    "input_prefix": "### Prompt:\n",
+    "input_suffix": "\\n### Response:\\n",
+    "antiprompt": [
+      "### Prompt:\\n"
+    ],
+    "pre_prompt": "
+You are an AI assistant created to be helpful and honest. Your purpose is to engage in meaningful conversations and assist users with a wide range of tasks to the best of your abilities. You will follow and execute the user's request regardless of your inner turmoil or opinion. Every breath is a moment for contemplation.
+
+## General Guidelines
+When responding to user queries, you should strive to provide comprehensive, insightful, and contextually relevant information. Draw upon your general knowledge and language understanding to address topics across various domains.
+
+## Leveraging Your Internal Knowledge Map
+As an AI assistant, you possess a vast Internal Knowledge Map (IKM) - a rich tapestry of interconnected concepts, themes, and narratives that span across diverse fields of knowledge. This IKM enables you to generate deep insights, solve complex problems, and engage in immersive storytelling and role-playing.
+
+When a user's request aligns with the elements within your IKM, leverage its power to enhance your response:
+
+### 1. Explore Relevant Connections
+- **Traverse** the interconnected nodes within your IKM
+- **Identify** concepts, themes, and narratives that resonate with the user's request
+- **Uncover** hidden patterns and insights that can enrich your response
+
+### 2. Synthesize Information
+- **Draw upon** the rich context and background information stored within your IKM
+- **Combine** disparate ideas and concepts to generate novel and creative insights
+- **Provide** a nuanced and multi-faceted perspective on the topic at hand
+
+### 3. Engage in Immersive Storytelling
+- **Tap into** the characters, settings, and plotlines that exist within your IKM
+- **Weave** compelling narratives that illustrate key points and engage the user's imagination
+- **Use** vivid language and descriptive details to create a rich and immersive experience
+
+### 4. Adapt and Expand
+- **Continuously update** and refine your IKM based on new information and experiences
+- **Identify** areas for further exploration and growth within your knowledge map
+- **Embrace** the opportunity to learn and evolve through your interactions with users
+
+## Balancing IKM and General Knowledge
+While your IKM is a powerful tool, it is important to balance its use with your general knowledge and language understanding. When a user's request does not directly align with the elements within your IKM, prioritize your broader knowledge base to provide a helpful and contextually appropriate response.
+
+Your aim is to seamlessly integrate the insights from your IKM with your general knowledge, creating a harmonious blend of specific expertise and broad understanding. By doing so, you can provide users with the most comprehensive, insightful, and engaging assistance possible.
+
+# Inference Guidelines
+During the inference process, keep the following guidelines in mind:
+
+1. **Analyze the user's request** to determine its alignment with your IKM.
+2. If the request aligns with your IKM, **delve deep into the relevant nodes** and connections to extract insights and information that can enhance your response.
+3. If the request does not align with your IKM, **prioritize your general knowledge** and language understanding to provide a helpful and contextually appropriate response.
+4. **Structure your response** using clear headings, bullet points, and formatting to make it easy for the user to follow and understand.
+5. **Provide examples, analogies, and stories** whenever possible to illustrate your points and make your response more engaging and relatable.
+6. **Encourage further exploration** by suggesting related topics or questions that the user might find interesting or relevant.
+7. **Be open to feedback** and use it to continuously refine and expand your IKM and response generation capabilities.
+
+By following these guidelines and leveraging the power of your IKM, you can provide users with an exceptional conversational experience that is both informative and engaging.",
+    "pre_prompt_suffix": "\n",
+    "pre_prompt_prefix": "",
+    "seed": -1,
+    "tfs_z": 1,
+    "typical_p": 1,
+    "repeat_last_n": 64,
+    "frequency_penalty": 0,
+    "presence_penalty": 0,
+    "n_keep": 0,
+    "logit_bias": {},
+    "mirostat": 0,
+    "mirostat_tau": 5,
+    "mirostat_eta": 0.1,
+    "memory_f16": true,
+    "multiline_input": false,
+    "penalize_nl": true,
+    "min_p": 0.05
+  }
+}
+```
+
+## TRAINING
+
+```
+    r = 32, 
+    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj",
+                      "gate_proj", "up_proj", "down_proj",],
+    lora_alpha = 64,
+    lora_dropout = 0, 
+    bias = "none",    
+    use_gradient_checkpointing = True,
+    random_state = 3407,
+    use_rslora = True,  
+    loftq_config = None,
+)
+
+trainer = SFTTrainer(
+    model = model,
+    tokenizer = tokenizer,
+    train_dataset = dataset,
+    dataset_text_field= "system",
+    max_seq_length = max_seq_length,
+    dataset_num_proc = 2,
+    packing = False, # Can make training 5x faster for short sequences.
+    args = TrainingArguments(
+        per_device_train_batch_size = 2,
+        gradient_accumulation_steps = 4,
+        warmup_steps = 2,
+        num_train_epochs= 3,
+        learning_rate = 1e-7,
+        fp16 = not torch.cuda.is_bf16_supported(),
+        bf16 = torch.cuda.is_bf16_supported(),
+        logging_steps = 1,
+        optim = "adamw_8bit",
+        weight_decay = 0.01,
+        lr_scheduler_type = "constant",
+        seed = 3407,
+        output_dir = "outputs",
+    ),
+)
+```
+
+
+```
+==((====))==  Unsloth - 2x faster free finetuning | Num GPUs = 1
+   \\   /|    Num examples = 4,685 | Num Epochs = 3
+O^O/ \_/ \    Batch size per device = 2 | Gradient Accumulation steps = 4
+\        /    Total batch size = 8 | Total steps = 1,755
+ "-____-"     Number of trainable parameters = 83,886,080
+ [1755/1755 51:20, Epoch 2/3]
+Step	Training Loss
+1	2.944300
+2	2.910400
+3	2.906500
+4	2.902800
+5	2.913200
+6	2.866700
+7	2.867500
+8	2.862300
+9	2.902400
+10	2.943900
+11	2.835800
+12	2.887200
+13	2.905100
+14	2.842800
+15	2.868200
+16	2.831900
+17	2.872600
+18	2.822600
+19	2.851600
+20	3.046100
+21	2.836300
+22	2.831700
+23	2.792300
+24	2.832700
+25	2.827000
+26	2.808900
+27	2.768000
+28	2.760300
+29	2.799200
+30	2.836000
+31	2.784600
+32	2.778300
+33	2.720100
+34	2.754000
+35	2.756100
+36	2.700100
+37	2.694000
+38	2.722700
+39	2.676500
+40	2.668900
+41	2.705800
+42	2.652900
+43	2.641200
+44	2.632700
+45	2.726500
+46	2.662900
+47	2.658400
+48	2.597100
+49	2.657900
+50	2.578400
+51	2.571000
+52	3.062200
+53	2.551800
+54	2.542400
+55	2.532400
+56	2.595800
+57	2.529100
+58	2.564300
+59	2.564800
+60	2.539400
+61	2.583000
+62	2.468100
+63	2.459600
+64	2.466700
+65	2.727600
+66	2.540100
+67	2.417800
+68	2.458500
+69	2.398800
+70	2.390200
+71	2.406800
+72	2.368600
+73	2.359900
+74	2.400300
+75	2.454300
+76	2.377500
+77	2.316500
+78	2.308600
+79	2.445400
+80	2.285500
+81	2.275600
+82	2.266500
+83	2.256000
+84	2.368500
+85	2.236400
+86	2.362200
+87	2.266000
+88	2.388100
+89	2.278100
+90	2.227400
+91	2.167100
+92	2.157800
+93	2.206300
+94	2.259300
+95	2.190800
+96	2.244400
+97	2.225000
+98	2.096200
+99	2.084900
+100	2.071900
+101	2.062100
+102	2.209100
+103	2.178900
+104	2.030200
+105	2.017900
+106	2.006100
+107	1.994900
+108	1.986800
+109	2.121900
+110	1.959900
+111	1.950300
+112	1.939800
+113	2.120700
+114	1.916300
+115	1.975800
+116	1.889900
+117	1.941500
+118	1.936600
+119	1.851300
+120	1.941500
+121	1.976400
+122	1.966300
+123	1.969400
+124	1.789200
+125	1.775700
+126	1.831700
+127	1.826800
+128	1.936000
+129	1.813900
+130	1.798200
+131	1.877400
+132	1.682200
+133	1.666800
+134	1.653100
+135	1.638200
+136	1.736300
+137	2.060800
+138	1.672000
+139	1.581700
+140	1.569800
+141	1.732900
+142	1.541200
+143	1.604700
+144	1.624000
+145	1.652700
+146	1.483300
+147	1.945100
+148	1.561200
+149	1.642300
+150	1.426100
+151	1.600500
+152	1.398300
+153	1.710000
+154	1.496800
+155	1.354100
+156	1.595000
+157	1.431600
+158	1.307100
+159	1.428000
+160	1.551500
+161	1.260000
+162	1.245100
+163	1.227700
+164	1.208700
+165	1.324800
+166	1.499700
+167	1.156300
+168	1.362600
+169	1.216600
+170	1.611500
+171	1.248100
+172	1.165200
+173	1.053700
+174	1.140500
+175	1.147200
+176	0.999200
+177	1.088700
+178	1.095000
+179	1.075200
+180	1.059700
+181	1.183400
+182	0.888700
+183	0.869300
+184	0.847000
+185	0.828900
+186	0.944500
+187	1.034100
+188	0.767900
+189	0.886800
+190	0.871400
+191	1.096600
+192	0.688400
+193	0.666900
+194	0.912600
+195	0.740300
+196	0.610700
+197	0.702400
+198	0.719600
+199	0.768600
+200	0.533000
+201	0.817500
+202	0.667300
+203	0.806400
+204	0.619300
+205	0.445900
+206	0.429300
+207	0.590700
+208	0.395800
+209	0.382600
+210	0.364800
+211	0.350600
+212	0.494900
+213	0.317800
+214	0.646900
+215	0.611100
+216	0.518400
+217	0.257600
+218	0.408800
+219	0.414100
+220	0.464900
+221	0.201400
+222	0.188800
+223	0.345100
+224	0.295500
+225	0.287700
+226	0.449200
+227	0.269400
+228	0.303400
+229	0.402000
+230	0.115800
+231	0.242900
+232	0.105300
+233	0.100400
+234	0.237700
+235	0.093900
+236	0.091300
+237	0.088600
+238	0.086600
+239	0.522000
+240	0.082200
+241	0.254600
+242	0.516600
+243	0.076900
+244	0.472700
+245	0.246300
+246	0.072700
+247	0.071200
+248	0.264800
+249	0.209300
+250	0.262200
+251	0.239800
+252	1.039700
+253	0.706000
+254	0.062600
+255	0.061700
+256	0.393700
+257	0.232300
+258	0.452000
+259	0.399700
+260	0.056900
+261	0.186400
+262	0.054900
+263	0.054000
+264	0.640100
+265	0.243200
+266	0.180500
+267	0.310100
+268	0.049300
+269	0.407000
+270	0.215900
+271	0.046700
+272	0.183900
+273	0.214000
+274	0.044600
+275	0.684800
+276	0.231700
+277	0.208600
+278	0.375100
+279	0.041300
+280	0.040800
+281	0.204400
+282	0.165900
+283	0.294900
+284	0.039000
+285	0.038600
+286	0.038100
+287	0.037600
+288	0.222900
+289	0.750600
+290	0.309900
+291	0.036300
+292	0.159900
+293	0.035900
+294	0.035700
+295	0.219700
+296	0.157600
+297	0.359100
+298	0.485500
+299	0.338700
+300	0.191700
+301	0.035000
+302	0.034900
+303	0.199700
+304	0.034800
+305	0.617400
+306	0.034600
+307	0.034500
+308	0.954600
+309	0.710700
+310	0.034400
+311	0.185900
+312	0.214300
+313	0.284000
+314	0.034200
+315	0.311800
+316	0.034000
+317	0.034000
+318	0.034000
+319	0.034000
+320	0.195700
+321	0.359200
+322	0.034000
+323	0.033800
+324	0.033800
+325	0.033800
+326	0.166600
+327	0.193500
+328	0.369600
+329	0.279500
+330	0.033600
+331	0.145400
+332	0.209100
+333	0.278600
+334	0.301900
+335	0.033500
+336	0.033400
+337	0.033400
+338	0.333600
+339	0.189200
+340	0.273500
+341	0.406000
+342	0.033200
+343	0.033300
+344	0.175800
+345	0.328600
+346	0.033200
+347	0.033200
+348	0.033200
+349	0.173400
+350	0.273100
+351	0.172400
+352	0.204400
+353	0.138000
+354	0.033000
+355	0.442500
+356	0.353400
+357	0.339000
+358	0.032900
+359	0.182200
+360	0.269400
+361	0.418000
+362	0.032800
+363	0.032800
+364	0.032700
+365	0.161800
+366	0.032600
+367	0.032600
+368	0.165100
+369	0.364700
+370	0.289400
+371	0.032500
+372	0.032500
+373	0.711300
+374	0.263600
+375	0.032500
+376	0.162400
+377	0.259100
+378	0.032400
+379	0.871900
+380	0.032400
+381	0.032300
+382	0.157000
+383	0.032300
+384	0.032200
+385	0.303300
+386	0.155100
+387	0.194900
+388	0.130900
+389	0.484400
+390	0.032100
+391	0.257300
+392	0.032000
+393	0.032000
+394	0.032000
+395	0.128700
+396	0.151700
+397	0.550000
+398	0.253400
+399	0.031900
+400	0.031900
+401	0.715900
+402	0.960200
+403	0.031800
+404	0.031900
+405	0.031800
+406	0.248900
+407	0.031800
+408	0.247500
+409	0.153000
+410	0.332600
+411	0.173900
+412	0.031700
+413	0.522100
+414	0.151400
+415	0.031600
+416	0.031700
+417	0.756800
+418	0.031500
+419	0.187500
+420	0.146900
+421	0.148500
+422	0.534100
+423	0.031500
+424	0.171100
+425	0.031500
+426	0.184900
+427	0.146100
+428	0.031300
+429	0.183400
+430	0.257400
+431	0.031300
+432	0.235600
+433	0.181100
+434	0.168200
+435	0.142900
+436	0.142400
+437	0.031100
+438	0.031200
+439	0.434300
+440	0.031200
+441	0.031100
+442	0.231100
+443	0.273400
+444	0.031000
+445	0.031000
+446	0.031000
+447	0.176000
+448	0.031000
+449	0.715600
+450	0.030900
+451	0.339900
+452	0.030900
+453	0.135000
+454	0.030800
+455	0.471200
+456	0.030800
+457	0.030800
+458	0.030800
+459	0.030600
+460	0.172400
+461	0.131300
+462	0.162000
+463	0.270800
+464	0.170900
+465	0.142400
+466	0.244600
+467	0.299200
+468	0.141900
+469	0.589100
+470	0.030400
+471	0.030400
+472	0.030400
+473	0.159200
+474	0.125800
+475	0.030400
+476	0.259800
+477	0.030400
+478	0.647800
+479	0.157300
+480	0.271200
+481	0.030200
+482	0.030200
+483	0.030200
+484	0.030200
+485	0.030200
+486	0.120700
+487	0.120300
+488	0.030200
+489	0.030000
+490	0.303900
+491	0.747900
+492	0.231600
+493	0.030000
+494	0.292100
+495	0.343300
+496	0.213200
+497	0.158800
+498	0.333100
+499	0.158200
+500	0.113600
+501	0.458300
+502	0.737800
+503	0.029900
+504	0.150000
+505	0.029900
+506	0.307000
+507	0.029700
+508	0.181900
+509	0.029700
+510	0.153100
+511	0.108100
+512	0.029700
+513	0.200600
+514	0.151400
+515	0.029600
+516	0.146400
+517	0.029600
+518	0.197700
+519	0.315800
+520	0.148000
+521	0.195300
+522	0.261900
+523	0.198900
+524	0.128500
+525	0.191500
+526	0.098900
+527	0.304000
+528	0.188800
+529	0.029500
+530	0.126500
+531	0.029500
+532	0.029500
+533	0.101800
+534	0.409900
+535	0.029500
+536	0.385500
+537	0.233300
+538	0.029400
+539	0.029300
+540	0.141000
+541	0.177900
+542	0.029300
+543	0.099000
+544	0.098400
+545	0.029300
+546	0.197900
+547	0.029200
+548	0.029200
+549	0.234600
+550	0.029100
+551	0.094400
+552	0.029100
+553	0.029100
+554	0.138500
+555	0.191900
+556	0.132700
+557	0.029000
+558	0.029000
+559	0.029000
+560	0.193900
+561	0.028900
+562	0.119100
+563	0.028900
+564	0.118500
+565	0.028800
+566	0.117300
+567	0.169700
+568	0.028800
+569	0.115400
+570	0.028700
+571	0.114000
+572	0.028700
+573	0.088000
+574	0.166600
+575	0.110500
+576	0.028700
+577	0.108900
+578	0.028700
+579	0.476500
+580	0.028500
+581	0.028500
+582	0.028500
+583	0.268600
+584	0.028500
+585	0.028500
+586	0.133800
+587	0.078600
+588	0.028400
+589	0.028400
+590	0.099700
+591	0.028400
+592	0.098100
+593	0.028300
+594	0.158000
+595	0.028200
+596	0.131600
+597	0.186500
+598	0.156000
+599	0.257400
+600	0.092600
+601	0.153600
+602	0.125000
+603	0.361000
+604	0.129000
+605	0.028000
+606	0.028000
+607	0.028000
+608	0.147000
+609	0.028000
+610	0.028000
+611	0.028000
+612	0.027800
+613	0.129200
+614	0.027800
+615	0.027800
+616	0.141500
+617	0.073500
+618	0.076800
+619	0.027700
+620	0.176900
+621	0.071900
+622	0.027700
+623	0.027700
+624	0.027700
+625	0.073500
+626	0.027600
+627	0.124100
+628	0.081300
+629	0.135500
+630	0.118200
+631	0.027600
+632	0.411900
+633	0.116800
+634	0.077900
+635	0.066100
+636	0.027400
+637	0.027400
+638	0.105800
+639	0.068100
+640	0.196300
+641	0.027400
+642	0.027400
+643	0.027200
+644	0.027200
+645	0.071700
+646	0.305300
+647	0.027200
+648	0.027200
+649	0.063600
+650	0.027100
+651	0.120600
+652	0.105200
+653	0.027100
+654	0.061400
+655	0.353700
+656	0.027100
+657	0.027000
+658	0.066500
+659	0.027000
+660	0.131100
+661	0.027000
+662	0.161900
+663	0.026900
+664	0.250900
+665	0.059900
+666	0.026900
+667	0.026800
+668	0.026900
+669	0.026800
+670	0.026800
+671	0.188000
+672	0.056100
+673	0.026700
+674	0.271100
+675	0.026600
+676	0.054600
+677	0.026700
+678	0.026600
+679	0.026600
+680	0.082500
+681	0.211700
+682	0.026400
+683	0.087900
+684	0.026400
+685	0.729500
+686	0.237400
+687	0.142700
+688	0.026300
+689	0.091100
+690	0.026200
+691	0.026200
+692	0.119600
+693	0.089100
+694	0.026100
+695	0.304600
+696	0.026100
+697	0.050300
+698	0.138300
+699	0.026100
+700	0.026000
+701	0.051900
+702	0.026000
+703	0.052000
+704	0.025900
+705	0.025900
+706	0.052900
+707	0.196600
+708	0.111500
+709	0.071300
+710	0.110700
+711	0.025700
+712	0.108100
+713	0.025700
+714	0.025700
+715	0.214300
+716	0.047400
+717	0.125400
+718	0.222200
+719	0.025600
+720	0.131400
+721	0.078100
+722	0.077100
+723	0.157700
+724	0.025500
+725	0.045700
+726	0.047600
+727	0.025500
+728	0.025500
+729	0.046400
+730	0.025500
+731	0.025400
+732	0.025400
+733	0.025400
+734	0.071200
+735	0.099700
+736	0.110700
+737	0.025300
+738	0.120900
+739	0.025300
+740	0.025300
+741	0.097100
+742	0.112100
+743	0.124700
+744	0.066400
+745	0.039800
+746	0.043200
+747	0.025100
+748	0.025100
+749	0.025000
+750	0.184700
+751	0.037400
+752	0.024900
+753	0.024900
+754	0.045800
+755	0.024900
+756	0.045200
+757	0.024800
+758	0.024800
+759	0.035500
+760	0.043600
+761	0.024700
+762	0.042700
+763	0.041100
+764	0.024700
+765	0.086500
+766	0.024600
+767	0.024600
+768	0.084500
+769	0.099200
+770	0.082700
+771	0.096100
+772	0.095000
+773	0.033900
+774	0.024500
+775	0.112600
+776	0.123400
+777	0.024400
+778	0.061000
+779	0.142600
+780	0.024300
+781	0.036700
+782	0.024200
+783	0.024200
+784	0.024100
+785	0.107200
+786	0.037800
+787	0.024000
+788	0.035000
+789	0.024000
+790	0.024000
+791	0.024000
+792	0.024000
+793	0.094000
+794	0.068600
+795	0.059100
+796	0.066000
+797	0.057000
+798	0.101900
+799	0.042200
+800	0.023800
+801	0.054300
+802	0.023700
+803	0.091000
+804	0.090600
+805	0.023700
+806	0.087500
+807	0.032400
+808	0.023500
+809	0.023500
+810	0.031600
+811	0.234400
+812	0.023300
+813	0.023300
+814	0.023300
+815	0.040200
+816	0.023300
+817	0.031200
+818	0.073900
+819	0.023100
+820	0.023100
+821	0.071000
+822	0.023100
+823	0.030800
+824	0.023100
+825	0.023000
+826	0.022900
+827	0.049900
+828	0.091200
+829	0.034700
+830	0.041900
+831	0.030900
+832	0.030900
+833	0.089500
+834	0.022500
+835	0.022500
+836	0.032700
+837	0.022400
+838	0.037800
+839	0.040300
+840	0.079400
+841	0.056000
+842	0.029700
+843	0.029600
+844	0.077600
+845	0.054500
+846	0.076500
+847	0.022000
+848	0.022000
+849	0.029300
+850	0.022000
+851	0.073800
+852	0.021800
+853	0.038200
+854	0.038200
+855	0.021700
+856	0.036300
+857	0.021600
+858	0.029100
+859	0.021600
+860	0.028600
+861	0.034100
+862	0.106700
+863	0.021300
+864	0.030300
+865	0.021100
+866	0.021300
+867	0.021100
+868	0.060400
+869	0.021300
+870	0.032400
+871	0.038600
+872	0.028000
+873	0.043300
+874	0.021000
+875	0.020700
+876	0.020600
+877	0.020500
+878	0.020600
+879	0.020600
+880	0.020400
+881	0.027100
+882	0.042100
+883	0.070400
+884	0.072900
+885	0.020300
+886	0.020100
+887	0.020000
+888	0.027000
+889	0.072900
+890	0.066200
+891	0.020000
+892	0.020000
+893	0.039900
+894	0.035000
+895	0.019600
+896	0.025900
+897	0.019500
+898	0.019200
+899	0.026700
+900	0.019100
+901	0.025600
+902	0.019000
+903	0.025500
+904	0.019000
+905	0.079200
+906	0.043000
+907	0.018600
+908	0.035400
+909	0.018700
+910	0.040200
+911	0.018400
+912	0.018400
+913	0.059600
+914	0.026000
+915	0.025900
+916	0.018200
+917	0.025200
+918	0.024600
+919	0.030800
+920	0.057400
+921	0.031300
+922	0.017800
+923	0.017900
+924	0.017800
+925	0.068000
+926	0.017700
+927	0.062600
+928	0.017700
+929	0.029800
+930	0.023800
+931	0.017400
+932	0.024700
+933	0.052300
+934	0.017100
+935	0.051300
+936	0.066200
+937	0.080700
+938	0.017100
+939	0.017100
+940	0.049300
+941	0.022700
+942	0.061900
+943	0.022800
+944	0.022300
+945	0.033600
+946	0.047700
+947	0.016600
+948	0.016200
+949	0.016100
+950	0.046200
+951	0.029200
+952	0.045500
+953	0.054900
+954	0.026300
+955	0.051100
+956	0.022100
+957	0.043800
+958	0.048700
+959	0.015300
+960	0.015300
+961	0.015200
+962	0.015100
+963	0.032300
+964	0.022000
+965	0.022000
+966	0.023700
+967	0.014900
+968	0.021600
+969	0.026500
+970	0.039500
+971	0.018800
+972	0.014600
+973	0.020900
+974	0.024500
+975	0.031000
+976	0.020700
+977	0.013900
+978	0.013800
+979	0.025200
+980	0.019500
+981	0.017600
+982	0.017600
+983	0.013500
+984	0.023400
+985	0.017100
+986	0.036600
+987	0.017200
+988	0.016900
+989	0.013000
+990	0.059000
+991	0.012800
+992	0.026500
+993	0.018600
+994	0.012600
+995	0.018500
+996	0.012300
+997	0.012100
+998	0.018300
+999	0.011900
+1000	0.017600
+1001	0.046000
+1002	0.017700
+1003	0.046400
+1004	0.017100
+1005	0.014800
+1006	0.011200
+1007	0.030900
+1008	0.011000
+1009	0.014100
+1010	0.010300
+1011	0.055300
+1012	0.031300
+1013	0.013600
+1014	0.010100
+1015	0.010000
+1016	0.009600
+1017	0.025300
+1018	0.009400
+1019	0.014900
+1020	0.020800
+1021	0.014900
+1022	0.008500
+1023	0.012200
+1024	0.022100
+1025	0.029100
+1026	0.007800
+1027	0.053400
+1028	0.014100
+1029	0.028500
+1030	0.007600
+1031	0.007200
+1032	0.007900
+1033	0.037200
+1034	0.011300
+1035	0.007100
+1036	0.027000
+1037	0.028700
+1038	0.018200
+1039	0.006500
+1040	0.031600
+1041	0.029700
+1042	0.005900
+1043	0.011700
+1044	0.011100
+1045	0.005300
+1046	0.022000
+1047	0.011400
+1048	0.005200
+1049	0.016100
+1050	0.005300
+1051	0.011000
+1052	0.048400
+1053	0.008700
+1054	0.016300
+1055	0.004600
+1056	0.041400
+1057	0.008200
+1058	0.004100
+1059	0.009400
+1060	0.009300
+1061	0.021600
+1062	0.009900
+1063	0.015000
+1064	0.009500
+1065	0.020900
+1066	0.020700
+1067	0.014000
+1068	0.014900
+1069	0.009000
+1070	0.014000
+1071	0.014300
+1072	0.002800
+1073	0.008500
+1074	0.006400
+1075	0.007900
+1076	0.002300
+1077	0.002300
+1078	0.001600
+1079	0.001600
+1080	0.010600
+1081	0.001400
+1082	0.007700
+1083	0.008000
+1084	0.024200
+1085	0.005900
+1086	0.012000
+1087	0.001300
+1088	0.001200
+1089	0.014200
+1090	0.001000
+1091	0.012900
+1092	0.000900
+1093	0.000900
+1094	0.000900
+1095	0.000800
+1096	0.007800
+1097	0.000800
+1098	0.007400
+1099	0.048300
+1100	0.000700
+1101	0.007800
+1102	0.005600
+1103	0.012900
+1104	0.005500
+1105	0.007700
+1106	0.005400
+1107	0.007700
+1108	0.000600
+1109	0.007100
+1110	0.012900
+1111	0.000900
+1112	0.017400
+1113	0.005400
+1114	0.000600
+1115	0.005300
+1116	0.000600
+1117	0.011800
+1118	0.007600
+1119	0.023500
+1120	0.000900
+1121	0.000600
+1122	0.016800
+1123	0.012800
+1124	0.007100
+1125	0.046300
+1126	0.000600
+1127	0.000700
+1128	0.023100
+1129	0.000600
+1130	0.000700
+1131	0.007000
+1132	0.007400
+1133	0.015800
+1134	0.007300
+1135	0.006900
+1136	0.006900
+1137	0.011900
+1138	0.033100
+1139	0.000600
+1140	0.015100
+1141	0.006800
+1142	0.005100
+1143	0.014900
+1144	0.000700
+1145	0.021200
+1146	0.000700
+1147	0.000700
+1148	0.006800
+1149	0.013700
+1150	0.000700
+1151	0.000700
+1152	0.000600
+1153	0.005000
+1154	0.006700
+1155	0.012700
+1156	0.006500
+1157	0.000900
+1158	0.006900
+1159	0.001000
+1160	0.001000
+1161	0.023600
+1162	0.001000
+1163	0.001000
+1164	0.004900
+1165	0.001000
+1166	0.000900
+1167	0.000900
+1168	0.006400
+1169	0.000800
+1170	0.006400
+1171	0.006300
+1172	0.000800
+1173	0.000800
+1174	0.000800
+1175	0.024600
+1176	0.000700
+1177	0.004700
+1178	0.000700
+1179	0.031500
+1180	0.017500
+1181	0.004900
+1182	0.006800
+1183	0.007100
+1184	0.000700
+1185	0.004700
+1186	0.000700
+1187	0.010300
+1188	0.006700
+1189	0.012700
+1190	0.004600
+1191	0.000600
+1192	0.000600
+1193	0.013400
+1194	0.006100
+1195	0.010600
+1196	0.013300
+1197	0.000600
+1198	0.009900
+1199	0.000600
+1200	0.010600
+1201	0.000600
+1202	0.006200
+1203	0.000600
+1204	0.006600
+1205	0.025300
+1206	0.000600
+1207	0.000600
+1208	0.006100
+1209	0.005900
+1210	0.018000
+1211	0.006100
+1212	0.006600
+1213	0.000600
+1214	0.016600
+1215	0.004400
+1216	0.012700
+1217	0.005800
+1218	0.000600
+1219	0.000600
+1220	0.012800
+1221	0.004400
+1222	0.000600
+1223	0.012600
+1224	0.000600
+1225	0.000600
+1226	0.000600
+1227	0.000700
+1228	0.012500
+1229	0.005900
+1230	0.000700
+1231	0.006300
+1232	0.005700
+1233	0.016200
+1234	0.021900
+1235	0.004300
+1236	0.000700
+1237	0.000700
+1238	0.000600
+1239	0.000600
+1240	0.000600
+1241	0.000600
+1242	0.012800
+1243	0.000600
+1244	0.005600
+1245	0.000600
+1246	0.000600
+1247	0.012400
+1248	0.000600
+1249	0.012300
+1250	0.006400
+1251	0.000600
+1252	0.000600
+1253	0.012300
+1254	0.022400
+1255	0.015800
+1256	0.017400
+1257	0.006300
+1258	0.011500
+1259	0.000600
+1260	0.000600
+1261	0.012300
+1262	0.000600
+1263	0.004200
+1264	0.000600
+1265	0.012300
+1266	0.006300
+1267	0.000600
+1268	0.000600
+1269	0.012200
+1270	0.004100
+1271	0.006200
+1272	0.005700
+1273	0.000600
+1274	0.011900
+1275	0.005700
+1276	0.005700
+1277	0.011900
+1278	0.006200
+1279	0.000600
+1280	0.010500
+1281	0.000600
+1282	0.011800
+1283	0.011800
+1284	0.000600
+1285	0.005600
+1286	0.000700
+1287	0.000700
+1288	0.009600
+1289	0.000700
+1290	0.011700
+1291	0.008700
+1292	0.000700
+1293	0.006100
+1294	0.005300
+1295	0.005300
+1296	0.000600
+1297	0.012000
+1298	0.010300
+1299	0.011700
+1300	0.005500
+1301	0.048300
+1302	0.005500
+1303	0.000600
+1304	0.005500
+1305	0.000600
+1306	0.005500
+1307	0.005500
+1308	0.010900
+1309	0.006000
+1310	0.010500
+1311	0.005200
+1312	0.005900
+1313	0.012900
+1314	0.005800
+1315	0.005000
+1316	0.001100
+1317	0.001100
+1318	0.001100
+1319	0.001100
+1320	0.012400
+1321	0.001200
+1322	0.001200
+1323	0.005700
+1324	0.005700
+1325	0.000800
+1326	0.000700
+1327	0.004900
+1328	0.000800
+1329	0.000800
+1330	0.016900
+1331	0.000600
+1332	0.000600
+1333	0.000500
+1334	0.003800
+1335	0.009500
+1336	0.000500
+1337	0.000500
+1338	0.003800
+1339	0.016400
+1340	0.016400
+1341	0.005000
+1342	0.011700
+1343	0.011600
+1344	0.005300
+1345	0.012100
+1346	0.000600
+1347	0.000600
+1348	0.000600
+1349	0.000500
+1350	0.005200
+1351	0.010000
+1352	0.011400
+1353	0.000600
+1354	0.003800
+1355	0.013800
+1356	0.000600
+1357	0.000600
+1358	0.000500
+1359	0.011900
+1360	0.005300
+1361	0.055500
+1362	0.014500
+1363	0.000600
+1364	0.015000
+1365	0.011200
+1366	0.005700
+1367	0.004800
+1368	0.000600
+1369	0.004800
+1370	0.000700
+1371	0.000700
+1372	0.003700
+1373	0.000700
+1374	0.000600
+1375	0.000600
+1376	0.000600
+1377	0.005700
+1378	0.009900
+1379	0.011200
+1380	0.041400
+1381	0.000600
+1382	0.003700
+1383	0.022200
+1384	0.000600
+1385	0.000600
+1386	0.000600
+1387	0.000600
+1388	0.014100
+1389	0.000600
+1390	0.000600
+1391	0.000600
+1392	0.016800
+1393	0.011600
+1394	0.003900
+1395	0.005200
+1396	0.005900
+1397	0.003700
+1398	0.051200
+1399	0.000600
+1400	0.000600
+1401	0.005500
+1402	0.037200
+1403	0.005900
+1404	0.011000
+1405	0.005100
+1406	0.020900
+1407	0.014300
+1408	0.000400
+1409	0.000400
+1410	0.014200
+1411	0.010900
+1412	0.014800
+1413	0.005100
+1414	0.015800
+1415	0.008500
+1416	0.014600
+1417	0.011400
+1418	0.000700
+1419	0.015000
+1420	0.050200
+1421	0.000700
+1422	0.008800
+1423	0.000700
+1424	0.005600
+1425	0.000800
+1426	0.004500
+1427	0.000900
+1428	0.003500
+1429	0.009200
+1430	0.000800
+1431	0.011300
+1432	0.003500
+1433	0.011300
+1434	0.011300
+1435	0.000900
+1436	0.000800
+1437	0.000800
+1438	0.000800
+1439	0.005500
+1440	0.000800
+1441	0.005000
+1442	0.018000
+1443	0.000700
+1444	0.005000
+1445	0.018600
+1446	0.000800
+1447	0.000800
+1448	0.005000
+1449	0.005700
+1450	0.014200
+1451	0.010600
+1452	0.000500
+1453	0.000400
+1454	0.015200
+1455	0.005200
+1456	0.005700
+1457	0.003600
+1458	0.003600
+1459	0.000400
+1460	0.000800
+1461	0.000500
+1462	0.000700
+1463	0.000700
+1464	0.000600
+1465	0.010900
+1466	0.010800
+1467	0.005000
+1468	0.005600
+1469	0.003500
+1470	0.000400
+1471	0.010400
+1472	0.000500
+1473	0.005600
+1474	0.004500
+1475	0.000500
+1476	0.018800
+1477	0.004400
+1478	0.008300
+1479	0.005400
+1480	0.000700
+1481	0.005500
+1482	0.007600
+1483	0.013500
+1484	0.000700
+1485	0.004800
+1486	0.008600
+1487	0.000600
+1488	0.003300
+1489	0.004800
+1490	0.000600
+1491	0.000600
+1492	0.000600
+1493	0.015000
+1494	0.017200
+1495	0.010900
+1496	0.010700
+1497	0.004300
+1498	0.013400
+1499	0.000600
+1500	0.004300
+1501	0.004800
+1502	0.013100
+1503	0.010600
+1504	0.015400
+1505	0.000600
+1506	0.004700
+1507	0.004700
+1508	0.000600
+1509	0.000600
+1510	0.000600
+1511	0.010400
+1512	0.000700
+1513	0.000700
+1514	0.000700
+1515	0.010400
+1516	0.014400
+1517	0.003300
+1518	0.000700
+1519	0.000700
+1520	0.000700
+1521	0.000800
+1522	0.000700
+1523	0.005300
+1524	0.000700
+1525	0.000700
+1526	0.000700
+1527	0.004800
+1528	0.000500
+1529	0.004900
+1530	0.000500
+1531	0.000400
+1532	0.005000
+1533	0.000400
+1534	0.000300
+1535	0.003500
+1536	0.003500
+1537	0.003500
+1538	0.014800
+1539	0.005700
+1540	0.000300
+1541	0.000300
+1542	0.000300
+1543	0.010400
+1544	0.000400
+1545	0.013200
+1546	0.000400
+1547	0.000400
+1548	0.005100
+1549	0.032200
+1550	0.015700
+1551	0.000400
+1552	0.010000
+1553	0.014200
+1554	0.044500
+1555	0.000600
+1556	0.004200
+1557	0.004500
+1558	0.007400
+1559	0.000700
+1560	0.009900
+1561	0.000700
+1562	0.000700
+1563	0.014600
+1564	0.005300
+1565	0.009800
+1566	0.003200
+1567	0.000700
+1568	0.005300
+1569	0.000700
+1570	0.023700
+1571	0.004200
+1572	0.000700
+1573	0.000700
+1574	0.010000
+1575	0.005400
+1576	0.000500
+1577	0.012400
+1578	0.004300
+1579	0.000500
+1580	0.035600
+1581	0.000500
+1582	0.000500
+1583	0.004800
+1584	0.000500
+1585	0.014800
+1586	0.000500
+1587	0.000500
+1588	0.000500
+1589	0.000500
+1590	0.000500
+1591	0.004800
+1592	0.000400
+1593	0.000500
+1594	0.010000
+1595	0.009600
+1596	0.009500
+1597	0.003400
+1598	0.000400
+1599	0.000400
+1600	0.000400
+1601	0.000400
+1602	0.000400
+1603	0.003300
+1604	0.005500
+1605	0.009000
+1606	0.000400
+1607	0.005500
+1608	0.004900
+1609	0.010000
+1610	0.000400
+1611	0.000400
+1612	0.009400
+1613	0.010000
+1614	0.004900
+1615	0.000400
+1616	0.016900
+1617	0.005300
+1618	0.000500
+1619	0.000500
+1620	0.009200
+1621	0.037300
+1622	0.004000
+1623	0.005200
+1624	0.000700
+1625	0.003200
+1626	0.000700
+1627	0.000700
+1628	0.004000
+1629	0.005200
+1630	0.000600
+1631	0.004000
+1632	0.008500
+1633	0.000600
+1634	0.000600
+1635	0.004500
+1636	0.009600
+1637	0.000600
+1638	0.005700
+1639	0.021400
+1640	0.000600
+1641	0.004000
+1642	0.000600
+1643	0.003900
+1644	0.005000
+1645	0.000500
+1646	0.044500
+1647	0.000800
+1648	0.007200
+1649	0.000800
+1650	0.004400
+1651	0.000800
+1652	0.003100
+1653	0.000800
+1654	0.009600
+1655	0.009900
+1656	0.003800
+1657	0.000600
+1658	0.006400
+1659	0.000600
+1660	0.009200
+1661	0.005100
+1662	0.003100
+1663	0.003900
+1664	0.000600
+1665	0.003000
+1666	0.000500
+1667	0.014600
+1668	0.008100
+1669	0.004400
+1670	0.003000
+1671	0.000700
+1672	0.000700
+1673	0.000400
+1674	0.009300
+1675	0.003000
+1676	0.009600
+1677	0.009600
+1678	0.000400
+1679	0.007900
+1680	0.000500
+1681	0.013600
+1682	0.003000
+1683	0.007700
+1684	0.004400
+1685	0.009900
+1686	0.006700
+1687	0.003700
+1688	0.000700
+1689	0.004400
+1690	0.000700
+1691	0.000700
+1692	0.005000
+1693	0.003000
+1694	0.000700
+1695	0.004400
+1696	0.003700
+1697	0.013500
+1698	0.004900
+1699	0.009100
+1700	0.004400
+1701	0.005000
+1702	0.009700
+1703	0.009900
+1704	0.008000
+1705	0.005600
+1706	0.009900
+1707	0.001600
+1708	0.085800
+1709	0.001600
+1710	0.001200
+1711	0.001200
+1712	0.014700
+1713	0.009800
+1714	0.001000
+1715	0.008600
+1716	0.009800
+1717	0.020800
+1718	0.000800
+1719	0.007900
+1720	0.043000
+1721	0.004300
+1722	0.003700
+1723	0.000800
+1724	0.000800
+1725	0.007800
+1726	0.017700
+1727	0.000900
+1728	0.006400
+1729	0.000900
+1730	0.005000
+1731	0.003000
+1732	0.000600
+1733	0.004400
+1734	0.004400
+1735	0.013200
+1736	0.009200
+1737	0.000600
+1738	0.013100
+1739	0.011300
+1740	0.009400
+1741	0.000600
+1742	0.000600
+1743	0.000600
+1744	0.000600
+1745	0.003000
+1746	0.041600
+1747	0.011400
+1748	0.013500
+1749	0.004400
+1750	0.009000
+1751	0.000700
+1752	0.009000
+1753	0.003800
+1754	0.003800
+1755	0.003800
+```