1  # Documentation Quality Assurance System
  2  
  3  This directory contains automated tools for ensuring the quality and consistency of the Planar documentation.
  4  
  5  ## Overview
  6  
  7  The QA system consists of three main components:
  8  
  9  1. **Link Validation** - Validates all internal and external links
 10  2. **Code Example Testing** - Tests all Julia code examples for correctness
 11  3. **Content Consistency Validation** - Ensures template compliance and style consistency
 12  
 13  ## Tools
 14  
 15  ### 1. Link Validator (`simple_link_validator.jl`)
 16  
 17  Validates all internal links in the documentation to ensure they point to existing files.
 18  
 19  **Features:**
 20  - Scans all markdown files for internal links
 21  - Validates file existence for relative and absolute paths
 22  - Generates detailed reports with broken link locations
 23  - Provides health score based on link validity
 24  
 25  **Usage:**
 26  ```bash
 27  julia docs/scripts/simple_link_validator.jl
 28  ```
 29  
 30  ### 1.1 Comprehensive Link Validator (`validate_links_final.jl`)
 31  
 32  Advanced link validation tool that provides detailed analysis and improvement tracking.
 33  
 34  **Features:**
 35  - Validates both internal and external links
 36  - Categorizes failures by type (internal vs external)
 37  - Tracks improvement from baseline (original 502 failures)
 38  - Provides comprehensive statistics and success rates
 39  - Shows sample failures for debugging
 40  
 41  **Usage:**
 42  ```bash
 43  cd docs/scripts
 44  julia validate_links_final.jl
 45  ```
 46  
 47  ### 1.2 Quick Link Check (`check_links.jl`)
 48  
 49  Simplified link validation for quick checks during development.
 50  
 51  **Usage:**
 52  ```bash
 53  cd docs/scripts  
 54  julia check_links.jl
 55  ```
 56  
 57  **Output:**
 58  - Console summary with health score
 59  - Detailed text report at `docs/reports/link_validation_report.txt`
 60  
 61  ### 2. Code Example Review
 62  
 63  Code examples are now reviewed using manual AI-guided procedures instead of automated testing. See `docs/maintenance/ai-code-block-review-guide.md` for comprehensive review procedures that ensure code quality while reducing maintenance overhead.
 64  
 65  **Example Skipping:**
 66  Code examples are automatically skipped if they contain:
 67  - `# This is just an example`
 68  - `# Not executable`
 69  - `# Pseudo-code`
 70  - `SomeHypotheticalFunction`
 71  
 72  ### 3. Content Consistency Validator (`content_consistency_validator.jl`)
 73  
 74  Validates template compliance, style consistency, and content standards across all documentation.
 75  
 76  **Features:**
 77  - **Template Compliance**: Checks for required frontmatter fields
 78  - **Heading Structure**: Validates proper heading hierarchy
 79  - **Markdown Syntax**: Detects malformed links, unmatched backticks, etc.
 80  - **Content Standards**: Ensures required sections in getting-started files
 81  - **Style Consistency**: Checks for trailing whitespace, line length, etc.
 82  
 83  **Usage:**
 84  ```bash
 85  julia docs/scripts/content_consistency_validator.jl
 86  ```
 87  
 88  **Output:**
 89  - Console summary with consistency score
 90  - Detailed text report at `docs/reports/content_consistency_report.txt`
 91  
 92  **Validation Categories:**
 93  - **Errors**: Critical issues that must be fixed (broken links, syntax errors)
 94  - **Warnings**: Important issues that should be addressed (missing sections, template compliance)
 95  - **Info**: Minor style issues for consideration (line length, whitespace)
 96  
 97  ## Makefile Integration
 98  
 99  The QA tools are integrated into the documentation Makefile for easy execution:
100  
101  ```bash
102  # Run individual tools
103  make validate-links        # Link validation only
104  make test-code-examples    # Code example testing only
105  make validate-content      # Content consistency validation only
106  
107  # Run all QA checks
108  make qa-all               # All quality assurance checks
109  ```
110  
111  ## Reports
112  
113  All tools generate reports in the `docs/reports/` directory:
114  
115  - `link_validation_report.txt` - Link validation results
116  - `code_examples_test_report.txt` - Code example test results  
117  - `content_consistency_report.txt` - Content consistency validation results
118  
119  ## Continuous Integration
120  
121  These tools can be integrated into CI/CD pipelines to ensure documentation quality:
122  
123  ```yaml
124  # Example GitHub Actions workflow
125  - name: Validate Documentation
126    run: |
127      julia docs/scripts/simple_link_validator.jl
128      # Code examples use manual AI-guided review
129      julia docs/scripts/content_consistency_validator.jl
130  ```
131  
132  ## Configuration
133  
134  ### Link Validator Configuration
135  
136  - `DOCS_ROOT`: Documentation source directory (default: `docs/src`)
137  - `REPORT_DIR`: Report output directory (default: `docs/reports`)
138  
139  ### Code Example Tester Configuration
140  
141  - `DOCS_ROOT`: Documentation source directory
142  - `REPORT_DIR`: Report output directory
143  - `TEST_DIR`: Temporary directory for test execution (default: `docs/test_examples`)
144  
145  ### Content Validator Configuration
146  
147  - `DOCS_ROOT`: Documentation source directory
148  - `REPORT_DIR`: Report output directory
149  
150  ## Best Practices
151  
152  ### For Documentation Authors
153  
154  1. **Always specify language for code blocks:**
155     ```markdown
156     
157     ```
158     # Bad: No language specified
159     println("Hello, World!")
160     ```
161  
162  2. **Add frontmatter to all pages:**
163     ```yaml
164     ---
165     title: "Page Title"
166     description: "Brief description of the page content"
167     ---
168     ```
169  
170  3. **Use proper heading hierarchy:**
171     ```markdown
172     # Main Title (H1)
173     ## Section (H2)
174     ### Subsection (H3)
175     #### Sub-subsection (H4)
176     ```
177  
178  4. **Mark non-executable examples:**
179  
180  ### For Maintainers
181  
182  1. **Run QA checks before merging:**
183     ```bash
184     make qa-all
185     ```
186  
187  2. **Address errors immediately** - broken links and syntax errors should be fixed
188  3. **Consider warnings** - missing sections and template compliance improve user experience
189  4. **Monitor success rates** - aim for >90% link health and code example success rates
190  
191  ## Troubleshooting
192  
193  ### Common Issues
194  
195  1. **"No markdown files found"**
196     - Check that `DOCS_ROOT` points to the correct directory
197     - Ensure markdown files exist in the specified location
198  
199  2. **Code examples failing**
200     - Verify Julia environment is properly set up
201     - Check if examples require specific packages to be loaded
202     - Consider marking non-executable examples appropriately
203  
204  3. **Many broken links reported**
205     - Verify file structure matches link references
206     - Check for case sensitivity issues
207     - Ensure relative paths are correct
208  
209  ### Performance Considerations
210  
211  - Link validation may be slow for many external links
212  - Code example testing creates temporary files (automatically cleaned up)
213  - Large documentation sets may take several minutes to validate completely
214  
215  ## Future Enhancements
216  
217  Potential improvements to the QA system:
218  
219  1. **External Link Validation** - Check HTTP status of external links
220  2. **Image Validation** - Verify referenced images exist
221  3. **Cross-Reference Validation** - Ensure "See Also" sections are bidirectional
222  4. **Performance Monitoring** - Track documentation build and test times
223  5. **Integration Testing** - Test complete user journeys through documentation
224  6. **Accessibility Validation** - Check for accessibility compliance
225  7. **SEO Validation** - Verify meta descriptions, titles, and structure for search optimization