What do the Catholic Church consider constitutes intrinsically impossible things?

God is omnipotent, but He is also good and wise and he bound Himself with some promises, with respect for our free will, with His love for us etc. As Thomas Aquinas (cited by Andrew Leach) explains, God can do nothing that is not in His will, or what He doesn't want. God clearly doesn't want any sin, and also nothing that is really foolish and illogical (though many of His deeds seem illogical to us with our limited minds).

See this article for further reasoning on this.

As other answers state, Christianity is universal in believing that God is omnipotent - that he can do anything. "Probability" is not an issue here. God can cause a coin toss to come down heads (or tails) whenever he wants. Likewise he could cause ten coin tosses to come down heads, or a thousand, or a million, or whatever. It doesn't matter how 'unlikely' it is, God can do it.

There are some things that God does not do, because they would be against his character.

The intrinsically impossible things that you are probably thinking of sometimes arise when people start to talk about logically contradictory things, things that are by their very definition nonsense. This gives rise to the famous "can God create a rock too heavy for him to lift", which is just an attempt to talk about something whose very existence contradicts itself. As CS Lewis puts it, "A nonsense sentence doesn't make sense just because you put 'Can God' in front of it.

I believe the only addition to the excellent answer by @AndrewLeach's answer is [God] cannot deny himself [e.g. sin, break his promises or oaths], the thing we do when we sin. According to Scripture and therefore Catholic Teaching, this appears to be the ONLY thing that God CANNOT do.

What can God really NOT do?

"Nonsense remains nonsense even if we talk it about God." C.S. Lewis: The Problem of Pain | Cf. This AndreasBlass' answer.

Is there really a limit to God's power? What does the Bible say? What does the Church teach?

The Bible:

1. "For with God nothing will be impossible.”
2. "Is anything too hard[wonderful] for the LORD?"
3. But thou art merciful to all, for thou canst do all things[.]
4. Our God is in the heavens; he does whatever he pleases.
5. [If] we are faithless, he remains faithful — for he cannot deny himself.

To me, according to the Bible, and therefore what the Church must teach [CCC 268-278], nothing is impossible for God and the only thing he cannot do, is deny himself.

Whence then comes Nonsense remains nonsense even if we talk it about God (cannot make a square/circle, make a stone too big for him to carry, etc.)?

From Christian reasoning, thinking, theology, and philosophy.

On the contrary, It is said: "No word shall be impossible with God" (Luke 1:37).

I answer that, All confess that God is omnipotent; but it seems difficult to explain in what His omnipotence precisely consists: for there may be doubt as to the precise meaning of the word 'all' when we say that God can do all things. If, however, we consider the matter aright, since power is said in reference to possible things, this phrase, "God can do all things," is rightly understood to mean that God can do all things that are possible; and for this reason He is said to be omnipotent. Now according to the Philosopher (Metaph. v, 17), a thing is said to be possible in two ways. - Summa Theologica > First Part > Question 25 > 3. Is He omnipotent?

I have of course a tremendous admiration and respect for the Angelic Doctor, and most likely I have failed to grasp his argument. I only note here that his theological thinking has not made it to official Church teaching.

Can God therefore make a square/circle, make a stone so big he cannot lift?

The answer lies in AND. Before the incarnation and even after, God becoming man - “We stone you for no good work but for blasphemy; because you, being a man, make yourself God.” - and to die on the cross, is a stumbling block to Jews and folly to Gentiles.

Therefore, for the same God, as man, Simon of Cyre′ne was compelled to carry his cross.

What about square/circle?

These shapes are mathematical concepts developed to help in understanding physical reality. A string in the shape of a circle and the same string turned into a square shape, I would say easy for God.

What has led to this thinking?

In physics: the wave-particle duality of light in particular and of matter in general.

In Biology: egg-laying mammal and innumerable/countess examples in the animal kingdom like parthenogenesis, sex in crocodiles determined by the nest temperature of the eggs, etc.

Mysteries of Faith: Jesus: God-made-man; Jesus' Substantial presence in the Eucharist, the elect get to see God, etc.

Church treasures and artifacts: The Shroud of Turin; The image of Our Lady of Guadalupe.

Etc.

Conclusion:

For with God nothing will be impossible. He remains faithful — for he cannot deny himself. - I personally would be extremely careful to even think that there is even a tiny bit limit to God's unbounded/infinite power.

I see no where in Scripture or Church teaching Nonsense remains nonsense even if we talk it about God.

It depends upon your definition of 'intrinsically impossible.' There are two standard ways to answer the question.

The first relates to logical contradictions. God cannot make a round square not because he is incapable but because it is an inherent contradiction - the proposition makes no sense. The same is true for making a rabbit a giraffe or any other contradiction.

Second, it is traditional in the orthodox Christian faith to claim that God cannot violate his own character without ceasing to be God. Note that this argument is not saying that he is 'unable' to do these things (lie, be unjust, etc.) but that if he did he would cease to be God. In the Christian tradition, God is defined primarily by his holiness and thus his character. If he were to violate that character he would cease to be God. One of the most common verses cited for this is 1 John 4:8 where the Bible states that "God is love." Therefore, if he were to act without love he would cease to be God.

In this way, the second answer to your question is a continuation of the first: it is a contradictory statement to refer to God as doing something against his character because in committing such an action he would cease to be God. These sorts of contradictions (round square, God is love but acting without love, etc.) are the only impossibilities Christians adhere to when describing God.

Thomas Aquinas states:

All confess that God is omnipotent; but it seems difficult to explain in what His omnipotence precisely consists: for there may be doubt as to the precise meaning of the word 'all' when we say that God can do all things. If, however, we consider the matter aright, since power is said in reference to possible things, this phrase, "God can do all things," is rightly understood to mean that God can do all things that are possible; and for this reason He is said to be omnipotent.

... It remains therefore, that God is called omnipotent because He can do all things that are possible absolutely; which is the second way of saying a thing is possible. For a thing is said to be possible or impossible absolutely, according to the relation in which the very terms stand to one another, possible if the predicate is not incompatible with the subject, as that Socrates sits; and absolutely impossible when the predicate is altogether incompatible with the subject, as, for instance, that a man is a donkey.

It must, however, be remembered that since every agent produces an effect like itself, to each active power there corresponds a thing possible as its proper object according to the nature of that act on which its active power is founded; for instance, the power of giving warmth is related as to its proper object to the being capable of being warmed. The divine existence, however, upon which the nature of power in God is founded, is infinite, and is not limited to any genus of being; but possesses within itself the perfection of all being. Whence, whatsoever has or can have the nature of being, is numbered among the absolutely possible things, in respect of which God is called omnipotent. Now nothing is opposed to the idea of being except non-being. Therefore, that which implies being and non-being at the same time is repugnant to the idea of an absolutely possible thing, within the scope of the divine omnipotence. For such cannot come under the divine omnipotence, not because of any defect in the power of God, but because it has not the nature of a feasible or possible thing. Therefore, everything that does not imply a contradiction in terms, is numbered amongst those possible things, in respect of which God is called omnipotent: whereas whatever implies contradiction does not come within the scope of divine omnipotence, because it cannot have the aspect of possibility. Hence it is better to say that such things cannot be done, than that God cannot do them. Nor is this contrary to the word of the angel, saying: "No word shall be impossible with God." [cf. Luke 1:37] For whatever implies a contradiction cannot be a word, because no intellect can possibly conceive such a thing.

(Summa Theologica, First Part, Question 25, Article 3, "Whether God is omnipotent?"; emphasis added)

This is long and dense, but I've highlighted the important parts. Aquinas says the following:

1. God is able to do anything that's not logically impossible, because He gives things existence, and anything that is logically possible can ipso facto exist.
2. With respect to those things which are logically impossible, it makes more sense to say of them "This is something that we can't logically talk about" or "This is a statement that makes no sense", rather than "This is something that God can't do"; because the defect is not in God's abilities, but in our mind's ability to make sense out of the statement.

There seems to be nothing in the Catechism to directly back this up; but statements like "Nothing is impossible with God, who disposes his works according to his will. ... He is the Lord of the universe, whose order he established and which remains wholly subject to him and at his disposal" (Catechism of the Catholic Church, paragraphs 105–106) point to God's omnipotence as dealing with what actually does happen and what is logically possible.