/ core / attention / __pycache__ / compaction.cpython-314.pyc
compaction.cpython-314.pyc
  1  +
  2  �0gi�F�
  3  ���Rt^RIHtHt^RIHtHt^RIHtHtH	t	H
  4  t
  5  Ht^RIH
t
^RIt!RR]
4t]!RR	44t]!R
  6  R44t]!RR
44t!RR4t!RR4tR/RRllt]R8XEdu]!R4]!4t]P3RRRR7]P3RRRR7]P3RRRR7]P3RRRR7]P5R4]P3RRR R7]P7R.4]!R!4R0F,t]P;]4t]!R#]R$]P>24K.	R%RR&RR'R1/R%RR&RR'R2/R%RR&RR'R3/R%R"R&RR'R(/R%RR&RR'R)/.t ]PC] 4t"]!R*]"PG424]!R+4]"PHF3t%]!R#]%P8R$]%PLR,]%PNR-24K5	]!R.4R#R#)4u�
  7  Attention-Based Compaction
  8  
  9  "Attention is all you need" - applied to memory and context management.
 10  
 11  The insight: What you attend to should survive. What you ignore should fade.
 12  This is how human memory works. This is how transformer attention works.
 13  This is how Sovereign OS should work.
 14  
 15  Compaction is not housekeeping - it's the core intelligence of the system.
 16  The compression decisions ARE the priorities.
 17  
 18  Principles:
 19  1. High attention → slow decay → stays in context
 20  2. Low attention → fast decay → gracefully fades
 21  3. Unresolved → never decays → persists until closed
 22  4. Cross-session attractors → boosted → clearly important
 23  
 24  The compaction algorithm runs during BIRTH phase (nightly) but
 25  the attention signals accumulate continuously.
 26  )�	dataclass�field)�datetime�	timedelta)�Optional�List�Dict�Any�Tuple)�EnumNc�.�]tRt^tRtRtRtRtRtRt	Rt
 27  R#)	�
RetentionTierz&Tiers of retention based on attention.�core�hot�warm�cool�cold�N)�__name__�
 28  __module__�__qualname__�__firstlineno__�__doc__�CORE�HOT�WARM�COOL�COLD�__static_attributes__r��</Users/rcerf/repos/Sovereign_OS/core/attention/compaction.pyr
r
s��0��D�
 29  �C��D��D��Drr
c�ta�]tRt^'toRt^tRtRt]V3RlRl4t	]V3RlRl4t
 30  V3RltR	tVt
R
 31  #)�AttentionScorezAttention score for an item.�Fc� <�V^8�dQhRS[/#���return��float)�format�
__classdict__s"�r �__annotate__�AttentionScore.__annotate__3s���?�?��?rc�^�VP'dR#\P!4VP,
 32  P	4p\
 33  P!VP)V,R,4pRVP,p\RVPV,V,4#)z.Compute effective score with decay and boosts.��?i�Q)�
 34  unresolvedr�now�
last_attended�
total_seconds�math�exp�
 35  decay_rate�cross_session_boost�min�	raw_score)�self�elapsed�decay_factor�boosts&   r �effective_score�AttentionScore.effective_score2s}���?�?�?���<�<�>�D�$6�$6�6�E�E�G���x�x���� 0�7� :�U� B�C���d�.�.�.���3�����5��=�>�>rc� <�V^8�dQhRS[/#r%)r
)r*r+s"�r r,r-Bs���
&�
&�m�
&rc��VPpVP'd\P#VR8�d\P#VR8�d\P
 36  #VR8�d\P#\P#)z.Determine retention tier from effective score.皙�����?��?皙�����?)r>r0r
rrrrr)r:�scores& r �tier�AttentionScore.tierAsi���$�$���?�?�?� �%�%�%�
�S�[� �$�$�$�
�S�[� �%�%�%�
�S�[� �%�%�%� �%�%�%rc�n<�V^8�dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&S[;R&#)r&�item_idr9r6r2�attend_countr7r0)�strr)r�int�bool)r*r+s"�r r,r-'s_�����L����	�
 37  �����
�����$����rrN)rrrrrrJr7r0�propertyr>rF�__annotate_func__r�__classdictcell__�r+s@r r"r"'sG����&�
 38  �L�!$���J�
�?��?��
&��
&�7�rr"c�4a�]tRt^RtoRtRtV3RltRtVtR#)�CompactionDecisionz(A decision about how to compact an item.Nc�r<�V^8�dQh/S[;R&S[;R&S[;R&S[;R&S[;R&S[S[,;R&#)r&rIrF�action�reason�attention_score�content_summary)rKr
r)r)r*r+s"�r r,�CompactionDecision.__annotate__RsT�����L����	�
 39  
�K��
�K�
�����c�]�)�rr)	rrrrrrXrOrrPrQs@r rSrSRs����2�&*�O��rrSc�xa�]tRt^]toRt]!]R7t^t^t	]
 40  V3RlRl4tV3RlRltV3Rlt
RtVtR	#)
 41  �CompactionResultzResults of a compaction run.)�default_factoryc� <�V^8�dQhRS[/#r%r()r*r+s"�r r,�CompactionResult.__annotate__gs���4�4�5�4rc�b�VP^8XdR#VPVP,#)zHow much we compressed.r/)�bytes_before�bytes_after)r:s&r �compression_ratio�"CompactionResult.compression_ratiofs,������!�����$�"3�"3�3�3rc� <�V^8�dQhRS[/#r%�rK)r*r+s"�r r,r^ms��� � �� rc��/pVPF4pVPPpVPV^4^,W&K6	RVPPR42RVPR2RVPR
2R.p\VP44Fwr5VPRVR	V24K	R
 42  PV4#)zHuman-readable summary.zCompaction at �%Y-%m-%d %H:%Mz
  Processed: z itemsz  Compression: �.1%z
 43    By tier:z    �: �
 44  )�	decisionsrF�value�get�	timestamp�strftime�items_processedrb�sorted�items�append�join)r:�by_tier�drF�lines�counts&     r �summary�CompactionResult.summaryms��������A��6�6�<�<�D�#�K�K��a�0�1�4�G�M� �
 45  �T�^�^�4�4�5E�F�G�H��D�0�0�1��8��d�4�4�S�9�:��	
 46  ��"�'�-�-�/�2�K�D��L�L�4��v�R��w�/�0�3��y�y���rc�f<�V^8�dQh/S[;R&S[;R&S[S[,;R&S[;R&S[;R&#)r&rnrprkr`ra)rrLrrS)r*r+s"�r r,r^]sL���������	�
 47  �&�'�E����
���rrN)rrrrrr�listrkr`rarNrbryrOrrPrQs@r r[r[]sA����&�+0��*E�I��L��K�
�4��4� � �!�rr[c��a�]tRt^�toRtRV3RlRlltRV3RlRlltV3RlRltV3R	lR
 48  ltV3RlRlt	V3R
lRlt
 49  RV3RlRlltV3RlRltV3RlRlt
V3RlRltRtVtR#)�AttentionCompactora!
 50  Compacts items based on attention scores.
 51  
 52  The compactor:
 53  1. Takes attention signals from the tracker
 54  2. Computes retention scores for all items
 55  3. Decides what to keep, compress, archive, or remove
 56  4. Applies compression strategies per tier
 57  
 58  Run during BIRTH phase for daily compaction.
 59  Nc�R<�V^8�dQhRS[S[S[3,RS[S[S[3,/#)r&�decay_rates�tier_thresholds)rrKr))r*r+s"�r r,�AttentionCompactor.__annotate__�s1���&�&��#�u�*�%�&��c�5�j�)�&rc
 60  ��T;'g
RRRRRRRRRR	/VnT;'g	R
 61  RRR
RR/Vn/Vn\4Vn\4VnR#)�bullet皙�����?�episoderD�link�333333�?�sessiong�������?�defaultg333333�?rrBrrCrN)r�r��_scores�set�_unresolved�_attractors)r:r�r�s&&&r �__init__�AttentionCompactor.__init__�s}��'�
 62  �
 63  ��c��s��C��t��t�+
 64  ��� /� 
 65  � 
 66  ��3��C��C�3
 67  ���35���!$����!$���rc�0<�V^8�dQhRS[RS[RS[RR/#)r&rI�	item_type�	intensityr'N)rKr))r*r+s"�r r,r��s3���!<�!<��!<��!<��	!<�
 68  
 69  �!<rc� �VPPW PR,4pWP9dxVPV,p\RVPR,VR,,4Vn\
 70  P!4VnV;P^,
unM0\VVV\
 71  P!4^R7VPV&WP9dRVPV,nR#R#)z�
 72  Record an attention event for an item.
 73  
 74  Args:
 75      item_id: The item that received attention
 76      item_type: Type of item (for decay rate selection)
 77      intensity: How intense the attention was (0-1)
 78  r�r/rBrD)rIr9r6r2rJr�N)r�rmr�r8r9rr1r2rJr"r�r7)r:rIr�r�r6rEs&&&&  r �record_attention�#AttentionCompactor.record_attention�s����%�%�)�)�)�5E�5E�i�5P�Q�
 79  ��l�l�"��L�L��)�E�!�#�u����'<�y�3��'N�O�E�O�"*�,�,�.�E�����!�#��$2��#�%�&�l�l�n��%�D�L�L��!��&�&�&�8;�D�L�L��!�5�'rc�$<�V^8�dQhRS[RR/#�r&rIr'Nre)r*r+s"�r r,r��s���4�4�s�4�t�4rc��VPPV4WP9dRVPV,nR#R#)z.Mark an item as unresolved (will never decay).TN)r��addr�r0�r:rIs&&r �mark_unresolved�"AttentionCompactor.mark_unresolved�s6�������W�%��l�l�"�/3�D�L�L��!�,�#rc�$<�V^8�dQhRS[RR/#r�re)r*r+s"�r r,r��s���5�5�s�5�t�5rc��VPPV4WP9dRVPV,nR#R#)z)Mark an item as resolved (can now decay).FN)r��discardr�r0r�s&&r �resolve�AttentionCompactor.resolve�s6����� � ��)��l�l�"�/4�D�L�L��!�,�#rc�4<�V^8�dQhRS[S[,RR/#)r&�
attractor_idsr'N)rrK)r*r+s"�r r,r��s#���@�@�D��I�@�$�@rc��\V4VnVF-pW P9gKRVPV,nK/	R#)z)Set the current cross-session attractors.r�N)r�r�r�r7)r:r�rIs&& r �set_attractors�!AttentionCompactor.set_attractors�s4���}�-���$�G��,�,�&�<?����W�%�9�%rc�&<�V^8�dQhRS[RS[/#)r&rIr')rKr
)r*r+s"�r r,r��s���*�*��*�
�*rc�|�WP9d\P#VPV,P#)z#Get the retention tier for an item.)r�r
rrFr�s&&r �get_tier�AttentionCompactor.get_tier�s-���,�,�&� �%�%�%��|�|�G�$�)�)�)rc�R<�V^8�dQhRS[S[S[S[3,,RS[RS[/#)r&rr�dry_runr')rrrKr	rMr[)r*r+s"�r r,r��s6���0�0��D��c��N�#�0��0�
 80  �	0rc��\\P!4\V4R7p\	RV44VnVEF:pVR,pVP
RR4pVP
RR4pWPP9dRp\Pp	M:VPV,PpVPV,Pp	VPWYV4p
 81  VPPV
 82  4V
 83  PR	8Xd%V;P \V4,
unK�V
 84  PR
 85  8XgEKT;P \V
 86  P";'gR4,
unEK=	V#)z�
 87  Run compaction on a list of items.
 88  
 89  Args:
 90      items: List of dicts with 'id', 'type', 'content', 'size'
 91      dry_run: If True, don't actually modify anything
 92  
 93  Returns:
 94      CompactionResult with decisions and stats
 95  �rnrpc
 96  3�x"�TF0pVPR\VPRR444x�K2	R#5i)�size�content�N)rm�len)�.0�items& r �	<genexpr>�4AttentionCompactor.run_compaction.<locals>.<genexpr>�s5���"5�.3�d�#'�(�(�6�3�t�x�x�	�2�7N�3O�"P�"P�.3�s�8:�id�typer�r�r�r#�keep�compress)r[rr1r��sumr`rmr�r
rr>rF�_make_decisionrkrsrUrarX)r:rrr��resultr�rIr�r�rErF�decisions&&&        r �run_compaction�!AttentionCompactor.run_compaction�s8��"��l�l�n���J�
 97  ��
 98  "�"5�.3�"5�5����D��4�j�G������3�I��h�h�y�"�-�G��l�l�*���$�)�)�����W�-�=�=���|�|�G�,�1�1���*�*�7�'�B�H����#�#�H�-����&�(��"�"�c�'�l�2�"����J�.��"�"�c�(�*B�*B�*H�*H�b�&I�I�"�"�-�2�
rc�2<�V^8�dQhRS[RS[RS[RS[/#)r&rIrFr�r')rKr
rS)r*r+s"�r r,r�s3���<�<��<��<��	<�
 99  
100  �<rc��V\P8Xd\VVRRRR7#V\P8Xd,\VVRRVPV,P
101  R7#V\P8Xd>VPV4p\VVRRVPV,P
102  VR7#V\P8XdgVPV4p\VVR	R
103  VPPV\V^R\P!4R74P
104  VR7#\VVR
RRR7#)z'Make a compaction decision for an item.r�zUnresolved - must persistr/)rIrFrUrVrWzHigh attention - full fidelityr�zMedium attention - compressed)rIrFrUrVrWrX�archivezLow attention - archivedr�)rIr9r6r2�removez$No attention - candidate for removalr#)r
rrSrr�r>r�_compress_contentr�_minimal_summaryrmr"rr1)r:rIrFr�rys&&&& r r��!AttentionCompactor._make_decisionsC���=�%�%�%�%����2� #��
��]�&�&�
&�%����7� $���W� 5� E� E��
��]�'�'�
'��,�,�W�5�G�%���!�6� $���W� 5� E� E� '�
�
��]�'�'�
'��+�+�G�4�G�%��� �1� $��� 0� 0��.�#�q�S�"*�,�,�.�;�!�#�?� '�
105  �
106  
�&����=� #��
rc�&<�V^8�dQhRS[RS[/#�r&r�r're)r*r+s"�r r,r�Xs���P�P��P��Prc�`�\V4R8:dV#VR,R\V4RVRR2#)z*Compress content while preserving essence.i,:N��Nz... [z chars] ...Ni�����r��r:r�s&&r r��$AttentionCompactor._compress_contentXs;���w�<�3���N��$�-���c�'�l�^�;�w�t�u�~�>N�O�Orc�&<�V^8�dQhRS[RS[/#r�re)r*r+s"�r r,r�as���%�%��%��%rc�>�\V4^d8:dV#VR,R2#)z$Create minimal summary for archival.:N�dNz...r�r�s&&r r��#AttentionCompactor._minimal_summaryas&���w�<�3���N��$�-���$�$r)r�r�r�r�r�)NN)r�r/)F)rrrrrr�r�r�r�r�r�r�r�r�r�rrPrQs@r r~r~�su����
107  �&�&�<!<�!<�F4�4�5�5�@�@�*�*�0�0�d<�<�|P�P�%�%rr~c�la�]tRtRtoRtV3RlRltV3RlRltV3RlRltV3R	lR
108  ltRt	Vt
109  R#)
�DailyNoteCompactoriiz�
110  Applies compaction to the daily note.
111  
112  During BIRTH phase, this:
113  1. Reads the day's accumulated content
114  2. Applies attention-based compaction
115  3. Writes compressed version for tomorrow's context
116  4. Archives full version for retrieval if needed
117  c�,<�V^8�dQhRS[RS[RS[/#)r&�	compactor�	daily_dir�archive_dir)r~rK)r*r+s"�r r,�DailyNoteCompactor.__annotate__ts)���<�<�%�<��<��	<rc��^RIHpWnV!V4VnV!V4VnVPPRRR7R#)���PathT)�parents�exist_okN)�pathlibr�r�r�r��mkdir)r:r�r�r�r�s&&&& r r��DailyNoteCompactor.__init__ts=��	!�"���i������,��������t�d��;rc�&<�V^8�dQhRS[RS[/#)r&�dater')rr[)r*r+s"�r r,r��s���$�$��$�-=�$rc���VPR4pVPVR2,pVP4'g!\\P
118  !4^R7#VP
4pVPWB4pVPPV4pVPVR2,pVPV4VPV4pVPVR2,p	V	PV4V#)za
119  Compact a day's daily note.
120  
121  Args:
122      date: The date to compact
123  
124  Returns:
125      CompactionResult
126  z%Y-%m-%dz.mdr�z-full.mdz
-compacted.md)
ror��existsr[rr1�	read_text�_parse_sectionsr�r�r��
127  write_text�
_rebuild_note)
128  r:r��date_str�
129  daily_noter�rrr��archive_path�compacted_content�compacted_paths
130  &&        r �compact_day�DailyNoteCompactor.compact_day�s����=�=��,���^�^��
131  �#�&6�6�
132  �� � �"�"�#�"�,�,�.� !��
�
133  �&�&�(���$�$�W�7�����.�.�u�5���'�'�X�J�h�*?�?������(�!�.�.�v�6�����X�J�m�*D�D���!�!�"3�4��
rc	�R<�V^8�dQhRS[RS[RS[S[S[S[3,,/#)r&r�r�r')rKrrr	)r*r+s"�r r,r��s6���$�$��$��$�
134  �d�3��8�n�	�	$rc��.pRp.pVPR4F�pVPR4'dkV'dHVPRVRV2RRRVR	RPV4R
135  \	RPV44/4VR,P4p.pK�VPV4K�	V'dHVPRVRV2RRRVR	RPV4R
136  \	RPV44/4V#)zParse daily note into sections.Nrj�## r��:r��section�namer�r�:�NN)�split�
137  startswithrsrtr��strip)r:r�r�rr�current_section�current_content�lines&&&    r r��"DailyNoteCompactor._parse_sections�s���������M�M�$�'�D����u�%�%�"��L�L���
138  �!�O�+<�=��	���!�4�9�9�_�#=���D�I�I�o�$>� ?�"��#'�r�(�.�.�"2��"$���&�&�t�,�(�"��L�L���
139  �!�O�#4�5��	����4�9�9�_�5���D�I�I�o�6�7��
��rc�&<�V^8�dQhRS[RS[/#)r&r�r')r[rK)r*r+s"�r r,r��s��� � �$4� �� rc�R�R.pVPRVPPR4R24VPRVPR
R24VPF�pVP
140  R9gKVPPR	4R,pVPR
141  V24VP
142  R8XdVPR4M6VPR4TPVP;'gR
4VPR
4K�	RPV4#)z'Rebuild note from compaction decisions.z# Compacted Daily Note
143  z*Compacted at rg�*z*Compression ratio: rhz*
144  r�r�r�z*[HOT - full fidelity]*
145  z*[WARM - compressed]*r�rj)r�r������)
146  rsrnrorbrkrUrIr�rXrt)r:r�rwr��section_names&&   r r�� DailyNoteCompactor._rebuild_note�s���+�-��
���~�f�&6�&6�&?�&?�@P�&Q�%R�RS�T�U�
���+�F�,D�,D�S�+I��M�N��(�(�H����"6�6�'�/�/�5�5�c�:�2�>�����s�<�.�1�2��?�?�f�,��L�L�#<�>��L�L�#8�:��L�L��!9�!9�!?�!?�R�@����R� �)��y�y���r)r�r�r�N)rrrrrr�r�r�r�rrPrQs@r r�r�is4�����<�<�$�$�L$�$�L � rr�c�^�V^8�dQhR\R\R\\\3,/#)r&r�r�r')rKr
147  r~r�)r*s"r r,r,�s2��&�&��&��&���1�1�2�&rc�x�^RIHpVf\V!V4R,4p\4p\	W0V4pW43#)z�
148  Create the compaction system.
149  
150  Args:
151      daily_dir: Path to daily notes
152      archive_dir: Path for archives (defaults to daily_dir/archive)
153  
154  Returns:
155      (AttentionCompactor, DailyNoteCompactor)
156  r�r�)r�r�rKr~r�)r�r�r�r��daily_compactors&&   r �create_compaction_systemr�s=������$�y�/�I�5�6��"�$�I�(��{�K�O��%�%r�__main__z(=== Attention-Based Compaction Test ===
157  �idea_001r�g�������?)r�r�rB�idea_002rC�idea_003rD�idea_005r�zRetention tiers:�idea_004z  rir�r�r�zNo attention ideazUnresolved - must keeprjz
158  Decisions:z (�)z
159  'Attention is all you need')N)r
rrrra High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail High attention idea with lots of detail z�Medium attention idea Medium attention idea Medium attention idea Medium attention idea Medium attention idea Medium attention idea Medium attention idea Medium attention idea Medium attention idea Medium attention idea z_Low attention idea Low attention idea Low attention idea Low attention idea Low attention idea )(r�dataclassesrrrr�typingrrrr	r
160  �enumrr4r
r"rSr[r~r�rr�printr�r�r�r�rIr�rFrlrrr�r�ryrkrvrUrVrrr �<module>rsV���,)�(�3�3����D���'&�'&��'&�T�*�*��*�� � �� �Df%�f%�Rv �v �t&�2�z��	�
161  5�6�"�$�I����z�X���M�
���z�X���M�
���z�X���M�
���z�X���M����j�)�
���z�X���M����j�\�*�
162163  ��O���!�!�'�*��
��7�)�2�d�j�j�\�*�+�P�
164  �z�6�8�Y�8g�h�	
�z�6�8�Y�8U�V�	
�z�6�8�Y�8Q�R�	
�z�6�8�Y�8K�L�	
�z�6�8�Y�8P�Q�
�E��
%�
%�e�
,�F�	�B�v�~�~�� �
165  !�"�	�.��
�
�
��
��1�9�9�+�R����z��A�H�H�:�Q�7�8��
166167  )�*�Ur