zwpride-iquestlab commited on
Commit
ec3b40a
·
verified ·
1 Parent(s): 2ccfcac

Update modeling_iquestloopcoder.py

Browse files
Files changed (1) hide show
  1. modeling_iquestloopcoder.py +24 -13
modeling_iquestloopcoder.py CHANGED
@@ -1,17 +1,28 @@
1
- # Copyright 2024 IQuestLoopCoder Authors
2
- #
3
- # Licensed under the Apache License, Version 2.0 (the "License");
4
- # you may not use this file except in compliance with the License.
5
  """
6
- IQuestLoopCoder Model Implementation for HuggingFace.
7
-
8
- Loop model passes hidden states through the decoder multiple times:
9
- - Loop 1: Standard attention, stores K1, V1 for each layer
10
- - Loop 2+: Mixed attention with gated combination of:
11
- - A: Full attention with Loop1's KV (global context)
12
- - B: Sliding window attention with Loop2's KV (local, high-precision context)
13
- - Gate g = sigmoid(linear(Q)), per-head
14
- - Output = g * A + (1 - g) * B
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
  """
16
 
17
  import math
 
 
 
 
 
1
  """
2
+ Modified MIT License
3
+
4
+ Software Copyright© 2025 IQuest Research
5
+
6
+ Our only modification is that, if the Software (or any derivative works
7
+ thereof) is used for any of your commercial products or services, you shall
8
+ prominently display "IQuest Coder" on the user interface of such product or
9
+ service.
10
+ Permission is hereby granted, free of charge, to any person obtaining a copy
11
+ of this software and associated documentation files (the "Software"), to deal
12
+ in the Software without restriction, including without limitation the rights
13
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
14
+ copies of the Software, and to permit persons to whom the Software is
15
+ furnished to do so, subject to the following conditions:
16
+
17
+ The above copyright notice and this permission notice shall be included in all
18
+ copies or substantial portions of the Software.
19
+
20
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
21
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
22
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
23
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
24
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
25
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
26
  """
27
 
28
  import math