US citizen travelling with non-US citizen child who was denied boarding by KLM for our travel to USA

You are a US Citizen. Therefore; your minor child is automatically a US citizen and the airline should know this. The refusal was unreasonable even if Timitic had labeled her no-admit. The airline is simply wrong and can be sued for the full damages because her entry can't lawfully be refused by the US government.

It is not possible to drop US citizenship as a minor by any means. Until she is eighteen she has dual citizenship no matter what she signs, or you sign for that matter.

This is a matter of Constitutional law. It doesn't matter if the text of the written law requires the airline to refuse her; the Constitution says she is a citizen and therefore cannot be refused entry. Therefore the law is void in this situation as though it was never passed. The right to travel in general can only be restricted by a Constitutional amendment (note that while the law prohibits you traveling to North Korea, it cannot prohibit you from traveling from North Korea had you already been there).

While there is such a thing in the US as a no-fly list; getting on it requires being an actual risk. Under no circumstances is a pre-teen child on that list legitimately and you would have cause against the US government for both failure of due process and failure of due diligence in such a case; however from reading the question, that didn't happen.

There's two things that could have happened; and either way KLM failed pretty hard. Either the US govt kicked out your daughter's ESTA application, or KLM acted on their own and decided ESTA application was invalid. In the first case, the action of the US government agent is unlawful and KLM should have ignored it. In the second case; KLM has made a very bad assumption and is exposed to all of the resulting liability on that basis alone. The fact remains that the documentation in your possession at the time was plenty to enter the US had you arrived at a US border so the denied boarding was in error and doesn't have an excuse.

There's a lot going on here, and likely mistakes being made on multiple sides.

Firstly, your daughter holds US citizenship by birth, and this fact was seemingly not declared on her ESTA application, which technically makes the application (and thus the resulting ESTA) fraudulent. I'm not necessarily suggesting this was done deliberately, and it likely wasn't the reason for the issues you experienced, but it's a good example of how people can make honest mistakes - as you (or your wife) did in this case!

Secondly, as the check-in agent claimed, the ESTA verification is fully automated. Once the travelers passport details are entered/scanned, the computer checks with US Department of Homeland Security to confirm the passenger is allowed board the plane - which seemingly in this case was denied by the computer. Without further details it's difficult to know why this was - perhaps there was a typo on the booking or the ESTA application (eg, an incorrect birthday - D/M/Y rather than M/D/Y, or something like that which could easily be missed). Or perhaps the agent entered the passport number incorrectly. Or perhaps DHS did actually deny boarding for a reason they considered to be valid. The simple fact is, there's no way to tell.

Yes, there is an escalation process the agent can follow in this situation. But again, without further details it's impossible to know why they didn't.

As to your specific question around compensation, the simple fact is that NO, you are not due compensation - or at least not unless you are able to prove that the airline made a mistake and realistically I don't think you'll be able to (especially given the US citizenship/invalid ESTA angle, which whilst likely not directly relevant, still shows that there is the potential that the ESTA was not valid).

EU-261 which is the regulation that covers such compensation for "denied boarding" specifically defines "denied boarding as :

(j) "denied boarding" means a refusal to carry passengers on a flight, although they have presented themselves for boarding under the conditions laid down in Article 3(2), except where there are reasonable grounds to deny them boarding, such as reasons of health, safety or security, or inadequate travel documentation;

In this case, you were denied boarding due to what the airline claimed was "inadequate travel documentation" so - presuming they were correct in doing so - there is no compensation due.

You can of course claim they were not correct in that claim, but proving that is likely going to be impossible. They will claim the DHS told them not to board the passenger, so they didn't.

A lot of comments here relate to whether the child should have been permitted an ESTA as they are deemed to have acquired US citizenship at birth.

I would refer to the document at https://fam.state.gov/fam/07fam/07fam0080.html

Section 7 FAM 085 states that US citizens are required to enter and depart on their US passport (in case of dual nationality as with this child) BUT it then states that “although a consular officer may not issue a visa to an individual who has been determined to be a US citizen, if a non immigrant visa applicant has a possible claim to US citizenship but is unable or unwilling to obtain documents to establish that status…the visa officer may presume the applicant is an “alien”…” and if eligible for the visa can receive it prior to the determination of their citizenship status.

Therefore if the child hasn’t yet gone through the process of recognising that they are a US citizen (and not all children born to US citizens abroad are eligible! As there are conditions on how long the US citizen parent lived and was physically present in the US) then this code indicates an ESTA or other non immigrant visa is fine, until the CRBA is filed and the embassy abroad has confirmed the child is definitely a US citizen.