It would probably be best to begin with the LOCUS Commentary article linked above. It is the distilled result of many years of thought about Imaginative Literature.
How to create an alien
The problem with creating truly alien aliens or worlds lies in the old country saying: "It ain't what you don't know that hurts you; it's all them things you know that ain't so."
Larry Niven and Jerry Pournelle, the authors of THE MOTE IN GOD'S EYE, are a good example: two knowledgeable and thoughtful men who worked out a detailed background. The "Moties", we are told, are either right-handed or left-handed, and in right-handed Moties, the left side of the brain is bigger than the right.
Sounds reasonable, doesn't it? That's because of the unstated--and therefore unexamined--assumption. The reason that one side of our brains controls the other side of our bodies is because our spinal cord is given a half-twist; most of the nerves cross over to the other side in the medulla oblongata.
Well, this is not a mechanical or electronic requirement; it's merely an evolutionary quirk. There's no reason to suppose--though it seems reasonable to us!--that this quirk would occur on any other planet. Niven and Pournelle were guilty of a thoughtless assumption--the bane of originality.
It's hard think outside the box when you don't realize you're in one.
But the real problem with the Moties is even deeper. Why would we assume that an alien brain would resemble a human one? Why two hemispheres with a vertical division? Why not a series of horizontal divisions, dividing the brain into flukes, like liver packed into a box? (And what would be the psychology of such a species? Perhaps it wouldn't have our tendency to split concepts into opposing dichotomies: good/bad, right/wrong, black/white.)
The Velantians of Doc Smith (E. E. Smith, PhD.; GALACTIC PATROL and GRAY LENSMAN in the Lensmen series) had a brain like a stick with pairs of balls glued to it (in a head like a dragon with many eyes). Each of these lobes controlled one of a Velantian's eyes and one of his/her many hands. Good for Doc, who examined his assumptions!
(Well, many of them. In story after story, Doc's aliens of many shapes came in two sexes, male and female; and his female aliens acted an awful lot like human women as depicted by popular writers of the 1920s. Hard to escape all assumptions.)
Throughout its history, Science Fiction has given us endless planets that closely resemble Earth in terms of gravity, air composition, temperature range, and biochemistry. On such worlds, it was assumed, we would require no special protective gear. We'd be able to breathe the air comfortably, eat the animals and plants, and so on. It was assumed that on these alien worlds, bugs would be small and that they would be hard-shelled and therefore jointed. Advanced life would be vertebrate, with spinal cords, and that the most advanced life would be warm-blooded.
Not one of these assumptions is warranted.
We cannot even assume that we can breathe air with the composition and temperature of Earth's. Why? Because of another unexamined assumption: that on an Earth-like planet, the biochemistry will be very close to ours. DNA, RNA, and so on. Very likely alien biochemistry will involve DNA and RNA, because nucleic acids form readily in a "primitive" atmosphere. Once they have formed, RNA/DNA based life is very likely. Still, we cannot assume that alien life will be biochemically like ours.
Even if we choose to make this assumption, we still can't guarantee breathable air. Why not? Because when we make the assumption that alien life is DNA/RNA based, we automatically assume that it would use the same 4 bases for its DNA. To us, a natural assumption, but why would it? We know of no reason why DNA could not have been built up from other purines and pyrimidines, like xanthine or adenosine, or why it had to have 4 bases. On the contrary, code experts will tell you that 2 bases are the minimum (as in binary code, on which computers run), up to--well, is there an upper limit? Who can say how different alien life could be, just by using a different number of codons?
In any case, even if they use the same 4 bases as we do, they could still give us a nasty shock. Earthly DNA produces some deadly proteins, which unEarthly creatures would consider normal--even tasty. Pollen, say, and organic dust, innocuous to them, might affect us like rattlesnake venom in aerosol form.
If that's too extreme for you, consider urushiol, the complex of related compounds that make poison ivy so pleasant to touch. It's not at all unlikely that alien plant life would be rife with it. Even if it was safe to breathe it (I don't know), a planet where the dust in the air causes you to break out would give even a Jedi Knight pause.
On Earth, plant cell walls are made of cellulose--chains of glucose stuck together; one of the first "plastic" molecules, or polymers. Animals, on the other hand, don't have cell walls as such. We have flexible cell membranes composed of proteins and lipids. --Might an alien plant have cell membranes of proteins? Why not? Imagine a plant that yields, not linen like the flax plant, but silk. Rotting "wood" on such a world might smell like a mildewed army blanket. (What would you call it? It wouldn't be wood!) Here again, a serious possibility for allergic reactions.
What was that about a spinal cord? Well, chordates like ourselves are at a serious risk if the cord gets broken, or even pinched. How about a nervous system with two or three main trunks running parallel, and cross connected, like a ladder--or, better, like the Internet? Breaking one of the trunks would not be such a serious problem. The animal would be able to route signals around the break, and it might well survive. Not so on Earth.
Can cold-blooded creatures be advanced? Well, here we're on less firm ground. All chemical reactions speed up with increase in temperature, and that goes for biochemical reactions too. But warming the entire organism is very inefficient. Small animals like mice and voles eat a sizable percentage of their body weight each day, just to keep the fires burning. Lizards, not so much.>
So, would it be possible to design a living creature whose nervous system and core muscles remained warm enough for rapid reactions if needed, but whose other organs might be much cooler? Difficult. Maybe possible. I suspect such a creature would be able, on frosty mornings, to jump, to sprint short distances, but if pursued, it would have to warm up. I'm thinking of mice gobbling bugs and small lizards too torpid to get away. On the alien planet, the pursuers would also have the same "efficient" build. Or maybe the arms race would end with all of them warm-blooded. (Really large animals are all effectively warm-blooded anyway, as they don't have time to cool down overnight.)
In any case, the exercise is worth it, just for the practice in thinking outside the box.
Bugs that aren't small? Why not? Bugs on Earth once were huge, dragonflies as big as hawks (Meganeura), spiders as big as a person's head (Mesothelae), giant centipedes (Arthropleura), and so on--but they all lived in the Carboniferous Epoch, 280 to 350 million years ago. At that time, the oxygen content of the air was half again what it is now. Because arthropods have passive breathing systems--even the arachnids, despite their "lungs"--they can't get enough oxygen in our current atmosphere to grow big.
So why couldn't alien bugs develop the very simple muscle-lung systems it would take to grow to human size? How big would we be, if our ancestors hadn't developed muscle-powered lungs? Even mice need to breathe. (Here's another step outside the box: our lungs work on a batch process--in, then back out. But our intestines work on a pass-through process. How would an animal differ if its lungs worked in such a manner, like a heart?)
Bugs that aren't hard-shelled or jointed? Again, why not? Study the evolution of endo- and exo- skeletons for even an hour on the Internet, and you'll begin to see how differently things might have gone. In any case, this is an assumption that should be examined.
Once examined, this, combined with the idea of large bugs, plus healthy doubts that only vertebrate life can be "advanced" will give you a truly alien alien: A giant bug with a tough leathery integument, an endoskeleton, lungs, and a nonhuman shaped brain. He might well look more like a scorpion than a man. (I'd name him "Fred".)
So, you've now had some examples and you know basically what to do. How do you go about doing it? I don't really know how I do it. Perhaps you should take a walk with a three- or four-year-old, and watch the child. You stride along the sidewalk, or the path, oblivious to practically everything. But she sees the dead beetle, and the little ants who are carrying away pieces of it. She sees the tiny spider in its small web, watching the ants. She sees the plastic wheel broken off some toy (black with a silvery "hub cap" painted on; the other side doesn't look like a wheel at all, it's just a sort of grid).
Normally you'd tug at her hand and say, "Come along, don't dawdle," or the like. Teaching her that such things are irrelevant, unimportant, to be ignored, not to be seen. Just as you were taught when young not to see. As with unimportant dead beetles by the sidewalk, so were you taught not to see the assumptions you grew up with. Learning to see them now will not be easy.
It's hard to think outside a box you don't know you're in.