1  Copyright 2024 LinkedIn Corporation
 2  All Rights Reserved.
 3  
 4  Licensed under the BSD 2-Clause License (the "License").  See License in the project root for license information.
 5  
 6  This product includes software developed by LinkedIn Corporation.
 7  
 8  This product contains code derived from the following open source projects:
 9  
10  1. Unsloth
11     Copyright (c) 2023 Unsloth AI
12     Licensed under the Apache License, Version 2.0
13     Source: https://github.com/unslothai/unsloth
14     
15     The `calculate_settings` function to determine block size and warp is reused for Norm and MLP operations.
16     Modifications and additions were made to the RMS Norm implementation.
17  
18  2. Triton
19     Copyright (c) 2023 OpenAI
20     Licensed under the MIT License
21     Source: https://github.com/openai/triton
22     
23     Modifications were made based on Triton tutorials for the RMS Norm implementation.
24  
25  3. Efficient Cross Entropy
26     Copyright (c) 2023 Mohamed Malek
27     Licensed under the MIT License
28     Source: https://github.com/mgmalek/efficient_cross_entropy
29     
30     The idea of gradient-in-forward and chunking was used in the Linear Cross Entropy implementation.
31  
32  4. Flash Attention
33     Copyright (c) 2023 Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, Christopher Ré
34     Licensed under the BSD 3-Clause License
35     Source: https://github.com/Dao-AILab/flash-attention
36     
37     Optimization ideas such as tiling and recomputation were inspired by this work.
38  
39  5. AutoAWQ
40     Copyright (c) 2023 Casper Hansen
41     Licensed under the MIT License
42     Source: https://github.com/casper-hansen/AutoAWQ
43     
44     The design of the automodel was referenced from this project.
45  
46  6. llm.c
47     Copyright (c) 2023 Andrej Karpathy
48     Licensed under the MIT License
49     Source: https://github.com/karpathy/llm.c
50     
51     The design of end-to-end testing was referenced from this project.
52  
53  7. Tiny Shakespeare Dataset
54     Source: https://huggingface.co/datasets/karpathy/tiny_shakespeare
55     
56     This dataset is used to conduct convergence tests on mini models.
57  
58  For full license texts, please refer to the respective project repositories.