Ah Mun, as he was known, was legendary for using the best ingredients. He made his own coconut milk, it was said, and colored his sweets with fragrant pandan leaves instead of green food coloring. And he had a secret supplier of gula Melaka ('Malacca sugar'—Malaysia's version of coconut palm sugar), essential to sweets like the pandan-flavored glutinous rice flour balls filled with shredded coconut called onde-onde, and pulut inti, pretty little pyramids of coconut rice cloaked in banana leaf, tops left unwrapped to reveal a crown of shredded, palm sugar-moistened coconut.

Before I tasted Ah Mun's kuih I hadn't known that sugar could be anything but sweet. But I bit into an onde-onde I tasted caramel and smoke, while the pulut inti suggested butterscotch and coffee. Even after tasting three (okay, maybe four) of each, I didn't feel the usual saccharine slickness on the back of my tongue. I had no sugar high. And I wanted more gula Melaka. More, more, more. I was smitten. So began my love affair with Malaysia's ancient sweetener.

Long before Europeans introduced cane sugar to Southeast Asia (and South Asia), there was palm sugar. Made in almost every country in the region, palm sugar is produced by boiling down the liquid collected from the cut stem of immature palm inflorescences, or flower stalks. After the liquid is reduced it is poured into various molds—four- or five-inch lengths of bamboo, halved coconut shells, circles formed from strips of rattan.

Depending on the type of palm from which the liquid is collected and the means by which it is processed, palm sugar can evince hints of sourness, smoke, chocolate, caramel, butterscotch, or coffee, or some combination of those flavors. It can range in color from golden (like most Thai palm sugars) to dark brown or almost black (like Indonesian gula aren, which is made with liquid collected from the aren palm). In Malaysia palm sugar is made primarily with liquid collected from coconut palms. Gula Melaka is named after the the former Dutch, Portuguese, and British colonial trading port of Malacca, on peninsular Malaysia's southwest coast.

I can't imagine Malaysian cuisine without gula Melaka. Its complexity lends depth to sweets like Ah Mun's kuih, and richness to the beloved shaved ice treat cendol, a marriage of melted palm sugar, fresh coconut milk, pandan-flavored mung bean flour noodles and sometimes, red beans and/or coconut rice.

Malaysians love their tau foo fah: warm, fresh soft bean curd doused with melted 'black sugar' (as opposed to 'white' cane sugar—the tau foo fah's euphemism for palm sugar) and their dodol, a sort of caramel made by stirring gula Melaka and coconut milk continuously for hours over a low fire.

And the award for Malaysia's most ethereal palm sugar treat? That's a tie between kuih ketayap, ethereally light pandan-fragrant crepes rolled around shredded coconut and palm sugar, and kuih keria, chewy sweet potato donuts that are first fried and then—wait for it—boiled in bubbling palm sugar, which both penetrates and flavors the dough and cloaks it in a crackly caramel-flavored glaze.

Gula Melaka plays well in savory dishes too, from Malay-style pineapple curry to ayam pongteh, Malaccan Nyonya (Peranakan) braised chicken with potatoes. Kelantan's small Chinese community celebrates temple festivals by cooking pork and chicken with pounded garlic, toasted star anise, soy sauce, and local coconut palm sugar for hong bak, a dish that is also an essential for ancestor worship during Chinese New Year and Qingming celebrations.

In a Malaysian Hakka dish, pork trotters are simmered in local black vinegar, whose flavor is reminiscent of Worcestershire sauce, palm sugar, and lots of ginger. And where would the country's national breakfast of nasi lemak be without a healthy dab of spicy-sweet-fishy sambal tumis?

Perhaps gula Melaka's most endearing trait is that it's good for you, at least compared to most others: the unrefined sugar has one of the lowest glycemic indices of any sweetener. (A quality gula Melaka contains no added cane sugar.) The good stuff is soft enough to shave easily with a carrot peeler or a sharp knife, and evinces enough depth of flavor to invite eating straight, like candy.

Unfortunately, short of hopping a plane to Malaysia there's no way to procure the real thing. But Indonesian gula jawa, a close relative, is available online, though purists should note that while it's palm sugar, it's not necessarily made from coconut palm. For an easier-to-use sweet alternative, seek out the organic coconut palm sugar from Bali's Big Tree Farms, which is produced in a similar fashion to gula Melaka and comes granulated, not in dense blocks.