TLDR: Because Latin mostly first came to English through French, we picked up the French habit of pronouncing most Latin-derived words with ?ce? and ?ci? using /s/ or sometimes /?/, but never /k/. We also got into the habit of not changing spellings once established.

Below please find the short version; the long version exceeds the space allotted to a given post here.

Who can we blame this silliness on?

I do imagine that all this must be opaque to someone looking at this as it exists right now; that is, synchronically. That's because there is no synchronic explanation, only a diachronic one. It is only when examined under the long meandering path we call history can one understand not just city, center, century, cycling and car, camomile, counter, cabbage but also opaque and pique, scholar and chronic, Celtic and Caedmon, species and special, coelacanth and Caesar, and all the many more besides.

Blaming the Romans

The heart of the matter has its origin in how English still uses, with few alterations, an alphabet devised thousands of years ago by the Ancient Romans for Latin back around the seventh century ??, some twenty-four hundred years ago.

In this system, the letter ?c? always represented /k/, never /s/. This letter’s own name cē was therefore pronounced /ke?/. Since ?c? was always /k/, that means that Caesar was pronounced by its owner exactly how it’s spelled: /?kaesar/. Of course, things have changed a lot since the 100 ?? – 44 ?? when Caesar was alive.

Why didn't they spell his name Kaesar then? Because even though the Romans did originally have a letter ?k? named ka, appearing in a few words like kalendae, they really didn't get much use out of it. Having two letters indicate the same sound was poor design, and this led to confusion in writing between kalendae and calendae. Soon enough, ?k? would come to be banished for centuries with calendae winning out, which is why our own word deriving from it is calendar not *kalendar.

Under the influence of Christian (Roman) missionaries, scribes, and clerics, the Celtic languages of the British Isles came to be written using the Latin alphabet under the convention of using ?c? for /k/ always. Notice how we pronounce Celtic /?k?lt?k/. It's also why the Welsh word for the Welsh language itself is pronounced [k?m?raiɡ] and spelled Cymraeg: because ?c? is still always /k/ under Welsh orthographic conventions. They don’t have a letter ?k? because they didn’t need one.

Blaming the Irish

The Celtic peoples of Britain were literate in the Latin alphabet, but the waves of invading Saxons, Angles, and Jutes (and Franks and Frisians) were not. They wrote using runes, but by the time the seventh century had rolled around, the writers of Old English swapped out their old runes for the Latin alphabet which the now-Irish missionaries in turn brought them.

The English modified the alphabet a bit to compensate for things English ha but Latin did not, yet it was mostly a mapping of the same sounds to the same letters as the Romans used. When the English found a new sound, they invented a new letter, like ?t? for /θ/ or ??? for /?/.

Originally when one wrote a Latin letter ?c? in English, it always meant /k/ just as it always used to mean that in Latin itself. So the English poet Caedmon's name started off with just as much of a /k/ sound as did Caesar's.

So why do we pronounce Caedmon with a /k/ but Caesar with an /s/? Because Caedmon was English, not Roman. And all this mess we can blame on the Romans, or at least on those who came after them, meaning the French, who also came after us in back in 1066. And that's why this all got messed up.

Blaming the French

Because in another three hundred years or so after Caedmon you get have the Norman Invasion. The Normans spoke one of Latin’s daughter languages, Old French, which was the language of the conquerors not the commoners. As such, the conquerors' Latinate vocabulary insinuated itself into the commoners' Germanic language. Taking place over a few centuries, this was the impetus that moved Old English to Middle English: all the Latin injections.

The thing is, the Latin injected into English was not Caesar's Latin any longer. The French had had their way with its pronunciation by folding, spildling, and mutating it into something Caesar would likely never have recognized if he had had a chance to hear it spoken.

But the written language he might well have been able to figure out.

That's because even as the spoken language of Classical Latin turned into Vulgar Latin and thence to Proto-Western-Romance and finally to the Old French which the Conqueror brought with him to England's green and pleasant land, scholars long struggled to preserve the old spellings more in keeping with Caesar's pronunciation than with their own.

Although there were a great many of these, one particular sound change concerns us here. In early Romance, /k/ followed by a front vowel like /i/ or /e/ was unstable. Think how in today's English /t/ followed by the the semi-vowel /j/ is unstable: said quickly, hit you and did you naturally become affricates not stops. I’m sure you see how these things happen, given that.

Exactly what became of that /k/ which had an /e/ or an /i/ or /j/ after it depends on which particular Romance language we're talking about. There were all kinds of transition stages for these changes in pronunciation, including ones intermixed with other yod-triggered sound changes, but in the present-day state of these language's respective evolutionary paths, Italian got /t??/ there, Spanish got /θ/ (but /s/ in the south and in America), Galego-Portuguese got /s/ (but /θ/ lingers in the far north), and French got /s/ and at times /?/.

Blaming the Greeks

The ?y? situation seen in Latin words like cycle and cylinder is in some ways different but in others the same. Latin didn't even have a letter ?y? until it borrowed it from Greek, so these were originally Greek words. Those were a /k/ in Latin. Since that ?y? represented a rounded front vowel in Greek (/y/ is just rounded /i/), the same thing befell ?cy? as ?ce? ?ci?. How cycle came to have a very different vowel than cylinder has a story of its own that I will here forego for brevity's sake, as it has no bearing on your question.

Blaming the Scholars

So even though ?c? used to mean /k/, the current mess all comes down to how because Latin mostly first came to English through French, we picked up the French habit of pronouncing most Latin-derived words with ?ce? and ?ci? using /s/ or sometimes /?/, but never /k/.

We also got into the habit of not changing spellings once established in writing.

This cannot be stressed enough, because it explains almost all these questions. Speaking and writing have followed their own separate journeys down through hundreds, even thousands of years. They are two completely different things.

It would make our writing harder to understand if we lost all etymological connection to our language’s long, long history. But it might make people be able to read it without having to know that then-lost history.