RickyPark LtdRickyPark Ltd · 20 September 2026
A knowledge assistant depends on more than the model. Reliable sources, access rights, update procedures, and handling of missing answers all affect its usefulness. A focused collection with a clear purpose is usually easier to evaluate than importing every company document at once.
1. Start with the questions people actually ask
Ask intended users to list recurring questions about a process, service, or product. Identify the authoritative source for each answer. Where different documents disagree, have the business owner resolve the conflict before the material is introduced.
2. Assign an owner to each collection
Record the title, purpose, owner, update date, and validity of each document. Remove duplicates or clearly retire superseded versions. Check text recognition in scanned files and ensure important information in tables and images can be understood.
3. Establish access rules before importing content
Distinguish public, departmental, and restricted information. Users should retrieve only sources they are authorised to read. Test restrictions across departments. An instruction telling a model not to reveal information does not replace authentication and retrieval permissions.
4. Prepare questions and reference answers
Build an evaluation set from real working questions with sensitive details removed. Record a reference answer and supporting source passage for each. Include missing answers, ambiguous questions, and conflicting material. Business reviewers should evaluate correctness, not just fluent writing.
5. Show sources and uncertainty clearly
Let users trace answers to the original content. Where evidence is insufficient, the assistant should explain the gap and refer to an owner. Evaluate references themselves: a link does not automatically mean the source supports the answer.
6. Plan ongoing content maintenance
Define how documents are added and removed, how quickly changes take effect, and who reviews results. Use unanswered questions to identify content gaps. When a document is deleted or its permissions change, verify that the assistant no longer exposes the old content.
A preparation checklist
- Defined users and recurring questions
- Approved documents and accountable content owners
- Versions, validity dates, and access rights
- Evaluation questions with reference answers
- Update, removal, and quality-review procedures
HKPC’s enterprise AI survey highlights gaps between tool use, formal workflows, and governance. These preparation steps make content management and ownership part of the assistant project. Read the research source
Explore the related service