Chris Leigh-Jones
Veteran Member
- Joined
- Apr 10, 2021
- Messages
- 99
- Vessel Name
- Vanguard
- Vessel Make
- Naval Yachts XPM-78
I don't sleep much as night so in those idle hours tend to play on the internet. I have an interest in AI and how it may affect our world so thought I'd see if AI in the form of ChatGPT could fool the US Coast Guard by passing the 50 or 100T Masters test. I downloaded a bunch of trial tests from a USCG and plugged them in.......
Questions fell in to three groups on a wide variety of subjects.
1. Prescriptive answers where ChatGPT could access the rules and regurgitate the correct one. Like Google but without 20 pages of them. It did pretty well at these. "What is the correct color for a life ring?" would be an example. 10/10
2.Interpretive ones where it needs a bit of care. It did less well at these and quite often missed the mark. "What is the sequence of sound signals approaching two closed bridges in a narrow canal?" is an example.It got it partly correct so a 5/10 result.
3. Outlyers involving interpretation or less prescriptive answers. "what is the annual inspection required for a portable CO2 extinguisher" it basically just made up believable rubbish. So a 0/10 for that. COLREGS tripped it up quite often also.
I then left a chart out on the dining room table with questions and a soft pencil and some dividers. Nothing happened but I waited till 5 am.
So gents, though it did well in some respects and always provided a thorough answer, though I think the demise of the mariner lies beyond our lifetimes. Finally defeated by a pencil and pointy dividers!
Questions fell in to three groups on a wide variety of subjects.
1. Prescriptive answers where ChatGPT could access the rules and regurgitate the correct one. Like Google but without 20 pages of them. It did pretty well at these. "What is the correct color for a life ring?" would be an example. 10/10
2.Interpretive ones where it needs a bit of care. It did less well at these and quite often missed the mark. "What is the sequence of sound signals approaching two closed bridges in a narrow canal?" is an example.It got it partly correct so a 5/10 result.
3. Outlyers involving interpretation or less prescriptive answers. "what is the annual inspection required for a portable CO2 extinguisher" it basically just made up believable rubbish. So a 0/10 for that. COLREGS tripped it up quite often also.
I then left a chart out on the dining room table with questions and a soft pencil and some dividers. Nothing happened but I waited till 5 am.
So gents, though it did well in some respects and always provided a thorough answer, though I think the demise of the mariner lies beyond our lifetimes. Finally defeated by a pencil and pointy dividers!