Back in the Chinese Room

The Chinese Room, you will recall, is a thought experiment proposed by philosopher John Searle – and every AI student knows it well.

The conceit is that Searle is in a booth where he is passed Chinese script through a slot. Although he neither speaks nor reads Chinese he does, however, have extensive instructions as to how to react to incoming messages, and, by following them, is able to write and pass back a sensible reply, also in Chinese. In this way he is simulating an AI computer program that might appear to understand Chinese. And so the question is – does he in fact innocently understand Chinese? Or, if he doesn’t, does the room?

There is much to be said about this, but the part I was thinking about was the role played by document automation tools. They might be seen as being a bit like the Chinese Room. In a typical application, such a set up allows a person who may not be in full possession or mastery of the subject to, nonetheless, make a sensible reply to a question. So, for instance, this might be making a bid response where the prospective client is asking a raft of standard questions. The responder can use the document automation application to select and add the appropriate canned response. Perhaps the client is asking all bidders what is their sustainability policy. The responder might need to have a nuanced understanding of the parameters and issues to do with sustainability in order to compose a good response from scratch, but, with the document automation (Chinese Room) application, they can select and include the pre-written response – perhaps understanding neither that nor the original question.

I suppose the nightmare scenario is that the client isn’t a knowledgeable human either, but something more like an Applicant Tracking System (ATS) that uses “AI” to analyse CVs, and is just processing the replies like another Chinese Room. There would then be very little human understanding going on in the whole transaction, but perhaps something would be being checked automatically that might have been a less valuable pursuit for a human to spend time on?

Those are the questions, I think. It’s all valid if it makes time for people to really focus on their value-adding, differential knowledge, experience and ideas. The document automation piece is only really KM in the sense of being a strategy to make more space for knowledge. That’s where the knowledge is – not so much in the automation as in the space it makes for creativity.