Jack Lin’s family is in the dehydrated shrimp business, and as a young boy the Southern California-raised chef would snack on the sweet and leathery crustaceans until his belly bulged. But his fondest memory of dried seafood is hands down his father’s homemade XO sauce, a spicy, delightfully crunchy mix of dried scallops, shrimp, and chiles along with cured ham, oil, and seasonings that many Chinese cooks adore.

"Everybody in my neighborhood had their own homemade recipe," recalls Lin, who currently serves as the chef de cuisine at Oakland's Hopscotch, an upmarket Asian-American diner and cocktail bar famous for its yonsei oyster (Shigoku oysters served on the half shell with uni, ikura, and citrus-soy dressing) and buckets of fried chicken that will rival any in the Bay Area. The birds are marinated for days in buttermilk, Japanese karashi mustard powder, soy sauce, and fresh ginger, then dusted lightly in flour before getting fried to a sublime crispness.

Hopscotch works their own homemade XO sauce into a number of dishes, including a grilled corvina served with kale ohitashi and barley. “It sits in a pool of thickened soy dashi and we top the fish with the XO,” says the restaurant’s chef-owner Kyle Itani. “The XO permeates into the broth as it is eaten and really packs a punch.” Fragrant with a hint of the ocean, on the nose XO plays out like a milder fish sauce. But once added to vegetables and meat, it brings all sorts of emotions to the surface. Sweet, salty, savory. It’s a puzzler. But one thing is certain: You like it.

On a recent fall afternoon, we're in the kitchen with Lin as he tells us everything he knows about this mysterious Chinese condiment. First, the name: "It's a status thing," he suggests while frying two cups of shallots. The sauce has its roots in the great Cantonese kitchens of Hong Kong, with XO short for "extra old"—which in the world of cognac is reserved for the finest bottles. (Cognac is widely idolized by the HK elite.) So when an ingenious chef in the 1980s first slapped the "XO" label on his new creation, diners instantly knew that they were in for something decadent.

So why is XO sauce so special? "Dried scallops are expensive," Lin explains as he checks the temperature of his wok oil. And true, the best XO sauces are made with whole dried scallops, which are harvested from as far as Hokkaido and typically cost over $50 per pound. At the restaurant, Lin and his cooks reserve the extraneous tags pulled from the fresh Maine scallops they serve, and dry them for the sauce. When the dried scallops are steamed, and then fried, it's like eating bacon bits from the sea,, intensely rich, sweet, and salty. They serve as the linchpin in this involved recipe that requires a good amount of time for sourcing ingredients and standing over frothing wok oil.

So what else goes into XO sauce? "Everything is fried individually at a low temperature, and then incorporated with a little sugar and chicken stock at the end," says Lin. The shallots, which are fried to a light golden brown are reserved. Next goes the garlic. Sizzle. Brown. Reserve. The labor-intensive process is continued with a kitchen sink's worth of briny and funky East Asian products: mentaiko (spicy cod roe), shirasu (frozen baby anchovies), two kinds of minced chile, and the dried scallops. Then—and this is where it gets really interesting—two cups of bacon. "That's Chinese cooking, man!" says Lin. "Love thy pork. Love thy smoky pork fat."

Traditionally the sauce is made with Chinese Jinhua ham, but at Hopscotch they cure and smoke their own bacon, so the burnt ends are saved for the sauce. Once fried, the reserved bits are mixed with brown sugar, a pinch of salt, and a bit of chicken stock, and then combined with the cooled cooking oil (which is rich with the flavors of all the components).

This leads to the last question: What does XO taste like? The first thing that comes to mind is Umami, or naturally occurring MSG—magical seasoning that brings out the fifth dimension in food and keeps us craving more and more. "It kind of goes with anything," Lin says. He admits to eating it straight when he's drinking beer, and he buys every jar he can get his hands on at the Asian market. "Every version is a little different, and it's cool to see the variations." But there is no question that homemade is the best.

When Lin joined the Hopscotch opening team three years ago, Kyle was making his own version of XO, which both chefs now admit was a little haphazard. The Taiwanese-American Lin saw an opportunity to impress his Japanese-American boss and make a mark on the menu. “That wasn’t XO,” Lin notes of his boss’s version, which neglected to fry each component individually. (The idea behind frying individually is to ensure each component is cooked to perfection; seconds matter in the deep fryer. Also, by frying the more delicate ingredients first you preserve their individual flavor before the oil is seasoned by the rest of the ingredients.) Kyle was happy to acquiesce. “If it’s better, it’s better. I was excited to see Jack taking the initiative and bringing so much passion to his work”. Kyle speaks about Lin, who was recently promoted to chef de cuisine, with great pride.

Up until this point, the recipe hasn’t been written down—not even for internal use. The restaurant would run out, and Lin would rush to gather the ingredients (sometimes coming in on a day off) and fry up a few days’ worth. He paused for a moment before agreeing to share it with us, and we’re thrilled he did. The sauce, which we admit is a little more complicated than the average home condiment recipe, keeps for months in your refrigerator, and is well worth the effort. That is, if it even lasts for a couple months. Because, it won’t.

Get the recipe for Bacon XO Sauce »
Get the recipe for Stir-Fried Shrimp and Long Beans with XO Sauce »

Daniel Holzman is the Brooklyn-based chef and owner of The Meatball Shop. Matt Rodbard is a writer living in Brooklyn and the author of Koreatown: A Cookbook.