Unless I were in a hurry, I would never choose to turn to GPT to analyze or synthesize text. I have little trouble getting myself to do that, and well, often with pleasure.
Where I’ve begun to choose to turn to it is for research, basically to find me answers to questions; to assemble relevant texts which were otherwise scattered or inaccessible. That it tends to perform unwelcome synthesis in the process is something I hadn’t thought through: I could ask it not to.
But it highlights a gap between what the tool is and what I need. It’s a search engine with a natural language interface (which means the right series of queries produces results that look conversational).
What I need is a protocol droid. GPT isn’t enough. It needs a layer in its index which tags everything from the sentence up in terms of its truth (by context) and then at the very least a gatekeeper: some logic that checks query results for consistency and reports contradictions.
There are gestures toward this, now, but it seems to be just repeating what some source or preponderance of sources seems (to it) to agree, which is part of why it is so confidently wrong when it’s wrong, and then immediately concedes (with equal confidence) when contradicted. I get better responses if I express doubt, but even those are squishy underneath.