If the evaluation completes but shows no results, the playbook may not have rules applicable to this document type, or the document structure may not match expected patterns. Try a different playbook or verify your playbook rules are specific enough.
Playbooks Evaluation in the Word Add-In
When you apply a Playbook to a document in the Word Add-In, August evaluates your document against each rule and presents results you can act on. This guide explains the evaluation workflow, how to interpret results, and how to work with suggested edits.
Running a Playbook Evaluation
When you attach a Playbook and begin a review, August runs an evaluation against your defined rules:
Attach your Playbook in the Word Add-In sidebar.
Start the evaluation — August begins analyzing the document against your playbook rules.
Watch progress — A progress indicator shows the current phase of evaluation.
The progress bar advances through these phases as August works:
Getting ready — Initializing the evaluation
Reading the document — Parsing and understanding the document structure
Reviewing clauses — Checking each clause against playbook rules
Analyzing findings — Determining match status and identifying issues
Drafting edits — Generating suggested changes where applicable
Checking edits — Validating proposed changes
Finalizing — Preparing results for review
Complete — Evaluation finished, results ready
The progress indicator is optimistic and monotonic — it shows forward progress but is not a literal step counter. It estimates completion based on rules reviewed and edits processed.
You can stop an evaluation mid-run if needed. Click Stop to halt the process and review partial results. The Stop button and processing indicator are synchronized with the evaluation state — when you stop, the interface reflects that no further processing is occurring.
Understanding Evaluation Results
Each playbook rule produces a result with a match status:
Status | Meaning |
|---|---|
Satisfied | The clause matches your preferred position or an acceptable fallback. |
Fallback | The clause matches fallback language but not the preferred position. |
Not Acceptable | The clause contains language you flagged as unacceptable. |
Not Applicable | The rule doesn't apply to this document section. |
Results that need attention appear with a Needs attention indicator, helping you focus on rules that require review rather than satisfied matches.
Working with Suggested Edits
For rules with NotAcceptable or Fallback status, August can generate edit suggestions that bring the clause closer to your preferred position.
Viewing Edit Suggestions
Each rule result shows citations that link to specific document locations. Click a citation like [1] or [2] to jump directly to that chunk in your document.
When edit suggestions are available, they appear as a Suggested edit with a preview of the proposed change. You can:
Apply — Accept the suggested edit
Dismiss — Remove the suggestion from your list
Choosing a Different Position
By default, edit suggestions are generated against your Starting position — the preferred language defined in your playbook rule. For rules with fallback positions, you can choose to regenerate edits from a different position:
Click the position selector labeled Change playbook position.
Select from available options: Starting position, Fallback 1, Fallback 2, and additional fallbacks if present.
Click Suggest edits to regenerate edit suggestions based on your selected position.
This is useful when you want to propose a fallback-aligned edit rather than pushing for the preferred position — for example, in negotiations where the preferred position is unlikely to be accepted.
Changing the position regenerates only the pending edit suggestions for that rule. Applied edits and completed reviews are not affected.
Regenerating Suggestions
If regeneration fails — for example, if the system cannot find the rule data or cannot determine preferred language — you'll see an error message. Try again or select a different position. Dismissed suggestions are filtered out and won't reappear unless you regenerate.
Applying Changes
Once you've reviewed individual suggestions:
Apply individual edits — Accept or dismiss each suggestion directly in the rule detail view.
Apply all changes — Accept all pending suggested edits at once.
When you apply changes, they appear as tracked changes in your Word document, allowing you to review them in context before finalizing.
After applying, the status indicators update to reflect completed actions. Click Done when finished with a rule, or Close evaluation to exit the evaluation view entirely.
Adding Run Notes
When you start a playbook evaluation, you can provide run notes — optional guidance specific to that evaluation run. Run notes help tailor the review to the current context:
Special considerations for this particular document or deal
Negotiation context that affects which positions to prioritize
Client-specific preferences that differ from the standard playbook
Run notes are attached to the evaluation run, not saved permanently to the playbook itself. This lets you provide run-specific context without modifying your playbook rules.
Troubleshooting
Evaluation returns no results
Couldn't regenerate edits for the new position
Regeneration may fail if the selected position doesn't have sufficient language data, or if the rule structure doesn't support the requested position. Select a different position or check that your playbook rule has complete fallback definitions.
Edit dismissed unexpectedly
Dismissed suggestions are filtered out of your list. If you dismissed an edit by mistake, regenerate suggestions for that rule to see it again.
Processing indicator stuck or Stop button remains visible
If the processing indicator appears stuck or the Stop button remains visible after an evaluation finishes, refresh the Word Add-In sidebar. The evaluation state is synchronized with the processing UI, so after stopping or completing, the interface should return to a ready state. If the issue persists, close and reopen the document to reset the evaluation session.
What to Read Next
Playbooks Overview for creating and configuring playbook rules
Word Add-In Overview for installation and general usage
Was this helpful?