Were ancient (Greek) battles fought like duels between heroes?

While there are records of single combat in ancient warfare, it definitely appears to have been the exception, rather than the norm. However, we can be reasonably certain that the tradition can be traced all the way from the Bronze Age to the later classical period.

As you mentioned, the Iliad contains many references to individuals engaging in single combat in the Bronze Age. Now, as noted above in the comments, we must be cautious when using Homer as an historical source. However, we also have records from other Bronze Age civilisations from the Eastern Mediterranean. An example would be the Tale of Sinuhe from Ancient Egypt.

We should also remember that many details from Homer that were previously thought to be just for dramatic effect have since been confirmed by archaeology. A famous example is the boar's-tusk helmet given to Odysseus in book 10 of the Odyssey. Given that single-combat was attested in other cultures in the region, and certainly continued in later tradition (where we have better records), it seems reasonable to give Homer the benefit of the doubt on this one.

By the seventh (or perhaps even eighth) century BC, the Hoplite had become the standard heavy-infantry in the ancient Greek world. Hoplites were not professional soldiers, but rather were the citizen-soldiers of the ancient Greek City-states. Their primary weapon was a spear, and they fought in a formation known as a phalanx.

That said, Greek armies did include significant numbers of "support troops" - soldiers other than hoplites. This included cavalry forces, light infantry (Psiloi), javelin throwers (akontistai), slingers (sfendonitai) and archers (toxotai).

What is sometimes forgotten is that among these "support troops" were often professional "single-combat specialists". This was a feature of ancient Greek warfare examined by G. L. Cawkwell in his article Orthodoxy and Hoplites (registration required).

The traditional view of Roman armies is of vast legions fighting together in formation. However, we even find examples of single combat in battle from the Roman world. These can be found from the early Roman Republic through to the late Roman period.

In the fifth-century BC, the Roman general Aulus Cornelius Cossus defeated the King of the Veientes, Lars Tolumnius, in single combat (Livy, IV, 19).

In another example, in the Numantine War (143-133BC) we hear of Scipio Aemilianus engaging in single combat with the King of the Celtiberians.

Much later, at the Battle of Dara between Rome and Persia in 530AD, we find yet another record of single combat being fought between the main opposing armies:

"But one Persian, a young man, riding up very close to the Roman army, began to challenge all of them,[29-36] calling for whoever wished to do battle with him. And no one of the whole army dared face the danger, except a certain Andreas, one of the personal attendants of Bouzes, not a soldier nor one who had ever practised at all the business of war, but a trainer of youths in charge of a certain wrestling school in Byzantium. Through this it came about that he was following the army, for he cared for the person of Bouzes in the bath; his birthplace was Byzantium. This man alone had the courage, without being ordered by Bouzes or anyone else, to go out of his own accord to meet the man in single combat. And he caught the barbarian while still considering how he should deliver his attack, and hit him with his spear on the right breast. And the Persian did not bear the blow delivered by a man of such exceptional strength, and fell from his horse to the earth. Then Andreas with a small knife slew him like a sacrificial animal as he lay on his back, and a mighty shout was raised both from the city wall and from the Roman army."

Procopius of Caesarea - History of the Wars

Source

Cawkwell, G. L: Orthodoxy and Hoplites, The Classical Quarterly, Vol. 39, No. 2 (1989), pp375-389