When President Trump meets President Xi Jinping of China this week to discuss contentious trade issues, they will face each other in another nation that was once the United States’ main commercial rival, seen as a threat to American dominance.

But the competition between the United States and Japan, which hosts the Group of 20 summit this week for the first time, settled into a normal struggle among businesses after waves of American anxiety in the 1980s. Japan hit a decade of stagnation, and in 2010, China overtook it as the world’s second-largest economy.

There is no sign, though, that the rivalry between the United States and China will reach the same kind of equilibrium. For one thing, Japan is a democracy that has a military alliance with the United States, while China is an authoritarian nation that most likely seeks to displace American military dominance of the western Pacific. In China’s competition with the United States, a rancorous trade war has persisted for a year, and issues of national security are bleeding by the week into economic ones. Some senior American officials are pushing for “decoupling” the two economies.

The main elements in relations — economic and commercial ties — have become unmoored, and few agree on the future contours of the relationship or the magnitude of the conflicts.

For American officials, the stakes seem much higher now than in the race with Japan. Most economists estimate China will overtake the United States as the largest economy in 10 to 15 years. And some senior officials in Washington now view China as a steely ideological rival, where the Communist Party aims not only to subjugate citizens but to spread tools of authoritarian control globally — particularly surveillance, communications and artificial intelligence technology — and establish military footholds across oceans and mountains.

Though Mr. Trump incessantly praises Mr. Xi — he said they “will always be friends” — the idea of China as a dangerous juggernaut, more formidable than the Soviet Union, has become increasingly widespread in the administration. It was articulated by Secretary of State Mike Pompeo during a visit to the Netherlands, part of a weeklong trip across Europe this month in which he talked about China at each stop.

“China has inroads too on this continent that demand our attention,” he told a news conference in The Hague. “China wants to be the dominant economic and military power of the world, spreading its authoritarian vision for society and its corrupt practices worldwide.”

The National Security Strategy issued by the White House in December 2017 sounded the alarm: The United States was re-entering an era of great power competition, in which China and Russia “want to shape a world antithetical to U.S. values and interests.” But since then, Mr. Trump and cabinet officials, distracted by Iran and other foreign policy matters, have failed to outline a coherent strategy.

That has left administration officials struggling to piece together an approach to China that has elements of competition, containment and constructive engagement, none of them sharply focused.

Mr. Trump’s closest advisers on China are split on strategies. His top foreign policy officials, John R. Bolton and Mr. Pompeo, have pushed for tough policies, as has Peter Navarro, the trade adviser and creator of a polemical book and documentary film, “Death by China.” In the opposite camp are tycoons — among them Treasury Secretary Steven Mnuchin, Stephen A. Schwarzman and Steve Wynn.

Midlevel bureaucrats are formulating their own ideas. The view of a drawn-out ideological conflict was laid out in stark terms by Kiron Skinner, the head of policy planning at the State Department, in a talk in Washington on April 29.

“This is a fight with a really different civilization and a different ideology, and the United States hasn’t had that before,” she said. “The Soviet Union and that competition, in a way, it was a fight within the Western family.”

Now, she said, “it’s the first time that we will have a great power competitor that is not Caucasian.”

Many analysts have tried to discern whether the striking remarks point to a new policy direction. Officials say privately that is not the case.

While there has been bipartisan praise in Washington for the administration’s tougher line — with measures ranging from tariffs to sanctions of Chinese technology companies — critics say they see strategic ambiguity without the strategy.

“The economic and security and technological and even scientific components of the U.S.-China relationship are now being conflated,” said Jessica Chen Weiss, a professor of government at Cornell University who studies Chinese politics and nationalism. “What’s worrying to many is not being able to decipher different levels of risk and how far and how quickly the efforts to indiscriminately decouple the United States and China will go.”

That idea of decoupling rests on the premise that two economies so intertwined poses a significant security risk to the United States. The linking accelerated when China entered the World Trade Organization in 2001 and in recent years had seemed irreversible. But Mr. Trump’s hard-line trade advisers want the two nations to unwind their supply chains, which means some American businesses exit China, and others stop selling components to Chinese companies.

Mr. Trump is narrowly focused on cutting the trade deficit with China, which many economists say is not meaningful. But his imposition of tariffs and the general uncertainty around the economic relationship are forcing some American companies to rethink keeping operations in China. And putting Chinese companies, notably Huawei, the giant maker of communications technology, on what officials call an entity list to cut off the supply of American components is having an effect.

“After a long period of globalization and squeezing out economic efficiencies, you do see national security rising to the forefront,” said Daniel M. Kliman, director of the Asia-Pacific Security Program at the Center for a New American Security.

That has not gone unnoticed in China. This spring, with trade tensions rising, Chinese state-run television began showing old Korean War films depicting American aggression. Newspapers ran editorials on the war.

Wang Wen, executive dean of the Chongyang Institute for Financial Studies at the Renmin University of China, said in an interview that the new model for United States-China relations was “fight but not break.”

The fallout from the struggle is widening. Chinese security officers have arrested two Canadian men on charges of spying in apparent retaliation for the arrest in Canada of Meng Wanzhou, a top Huawei executive, on an extradition request by the United States. The F.B.I. has been canceling visas of Chinese scholars suspected of intelligence ties.

Some observers say they fear a new Red Scare.

“Rather than proclaiming a ‘whole of society’ threat from a hostile ‘civilization,’ U.S. officials would be wise to emphasize the value that immigrants from China and other countries have brought, while establishing policies to safeguard against theft of intellectual property,” Ms. Weiss said.

The case of Huawei is at the nexus of concerns in Washington over both Chinese economic dominance and security threats. The Trump administration has been pushing countries to bar Huawei from developing next-generation 5G communications networks, arguing it poses a national security risk. Huawei, a private company, denies the charge.

But the reluctance of even close allies to adopt a ban, except for Australia, shows how nations are unwilling to jeopardize their economic relations with China. That includes Japan, where the government has not issued a ban and is trying to strengthen ties with China in other ways — at the G20 in Osaka, Prime Minister Shinzo Abe plans to host a dinner for Mr. Xi.

The Trump administration has also been pushing countries to reject China’s Belt-and-Road infrastructure projects and what American officials call “debt diplomacy,” with mixed results.

Some American companies are trying to bypass the limits set by the Trump administration on their dealings with China. Semiconductor companies, for example, have found a legal basis for sidestepping the Commerce Department prohibition on selling components to Huawei.

But the administration itself sometimes pulls punches on China in the name of economic relations — a sign that the traditional foundation of the relationship still stands to a degree.

Since last year, the administration has debated imposing sanctions on Chinese officials for their role in interning one million or more Muslims in the Xinjiang region. Though Mr. Pompeo and other officials have pushed for the sanctions, the Treasury Department, led by Mr. Mnuchin, has opposed them for fear of derailing the trade talks. So the administration has taken no action.

China’s extraordinary human rights abuses in Xinjiang are one major reason many American officials have abandoned any notion of a future turn toward liberalism within the Communist Party.

For their part, Chinese officials have seized on the Trump administration’s actions to argue that the United States is trying to stop China’s rise. On Tuesday, People’s Daily, the official Communist Party newspaper, ran a commentary urging citizens to fight for the nation’s dignity.

“The Chinese people deeply understand that the American government’s suppression and containment of China is an external challenge that China must bear in its development and growth,” the paper said, “and it is a hurdle that we must overcome in the great rejuvenation of the Chinese nation.”