The problem

AI features are now appearing in almost every kind of business software. In many cases they are genuinely useful: drafting text, summarising records, pointing out gaps. But AI is also easy to overstate. A feature described as "AI-powered approval" or "automatic root-cause detection" suggests a level of reliability and responsibility that current AI systems cannot carry.

In professional fields such as health and safety, incident investigation and athlete development, the difference matters. Decisions in these areas affect people's safety, their reputation and sometimes their health.

Why it matters

AI models can produce fluent, confident output that is incomplete or wrong. They do not know the site, the people or the history unless that information is provided, and even then they can misread it. They cannot be held accountable for a decision.

When AI output is treated as a decision rather than an input, several risks follow:

  • Approvers rely on the AI and stop reading the permit carefully.

  • Investigators accept a suggested cause without testing it against evidence.

  • Users treat general guidance as personal medical or professional advice.

  • Responsibility becomes unclear: was it the person or the system that decided?

At the same time, refusing to use AI at all can mean missing real benefits. A well-placed suggestion or second-opinion review can catch something a busy professional overlooked.

A practical approach

A useful way to think about AI in professional software is to separate assistance from decision.

AI can reasonably:

  • Suggest things to consider, such as questions, possible gaps or exercises.

  • Review content and point out issues worth checking, as a second opinion.

  • Draft text that a person then reads, edits or rejects.

  • Summarise records to save time.

AI should not:

  • Approve work or declare that work is safe.

  • Make safety-critical decisions autonomously.

  • Confirm root causes or finalise an investigation.

  • Diagnose medical conditions or give medical advice.

  • Change records, plans or permissions without a person's action.

A few design principles help keep AI in its place. AI output should be clearly labelled. A person should explicitly accept a suggestion before it becomes part of a formal record. Formal records, approvals and issued reports should remain the work of accountable people. And the product should be honest about what its AI does and does not do.

The central principle is simple: AI assists; people decide.

What a good system should provide

Professional software that uses AI responsibly should:

  • Present AI output as advice, separate from formal decisions.

  • Keep approvals and sign-offs with authorised people.

  • Require human acceptance before AI suggestions enter formal records.

  • Avoid medical or diagnostic claims, and remove sensitive information that the AI does not need.

  • Be clear about which AI features are available now and which are still in development.

How Sospita applies this

Sospita uses AI in a small number of specific places, each with defined limits.

Sospita SafeGuard: AI permit review (available now). Before approval, an AI review of the permit content offers advisory recommendations. It does not approve permits or decide whether work is safe, and it should not replace professional judgement. Approval remains with the authorised workflow user groups. More on what a permit system should control is in What Should a Digital Permit to Work System Actually Control?.

Sospita Athletic Performance: AI Daily Coach and AI Monthly Athlete Evaluation (available now). The AI Daily Coach gives each signed-in user a short daily message with suggested exercises, chosen from a limited set of allowed exercises. It does not give medical advice, and medical and injury notes are removed before information is sent. The AI Monthly Athlete Evaluation, an optional add-on, produces a draft review for a coach to read; the coach decides whether to generate it and how to use it.

Incident Investigation & RCA: AI investigation support (coming soon). AI support for investigators has been built but is not yet available. When released, it will offer suggestions only. A person must accept each suggestion, causes suggested by AI remain candidates, and the AI will not confirm causes, assign action owners or write the issued report.

Sospita Risk Assessment and My Checklists do not use AI.

Limitations and human responsibility

Even advisory AI has limits. Its suggestions may be incomplete, irrelevant or wrong, and the absence of an AI warning does not mean there is nothing to worry about. Users should treat AI output as one input among several.

AI features also depend on third-party models. Their behaviour can change over time, and no AI review should be relied upon as a complete check.

Professional judgement, accountability and the duty of care remain with people: the approver who signs the permit, the investigator who confirms a cause, and the coach who works with an athlete. Software can support those people. It cannot take their place.