The Sulzberger family’s control of the New York Times Company came under attack by Morgan Stanley, the company’s fourth-largest shareholder, which said its reliance on two classes of stock means executives doesn’t have to answer to investors.

“While it may have at one time been designed to protect the editorial independence and the integrity of the news franchise, the dual-class voting structure now fosters a lack of accountability to all of the company’s shareholders,” New York-based Morgan Stanley Investment Management said in a statement yesterday.

New York Times, the third-largest newspaper publisher in America, should eliminate its Class B shares, which elect nine of 13 directors even though they represent less than 1% of the company’s market value, a managing director of the Morgan Stanley unit in London, Hassan Elmasry, said in the statement. The firm withheld votes for directors at New York Times’s annual meeting yesterday in protest of the two-tier system.

New York Times finds itself the target of growing activism among investors fed up with managers who fail to deliver rising earnings and share prices. Shareholders of Knight Ridder this year forced a sale of the company. Going public is a direct challenge to Chairman Arthur Sulzberger Jr., whose family purchased the New York Times newspaper in 1896.

Morgan Stanley owns 5.6% of New York Times and said it has held shares on behalf of clients since 1996. The firm said other institutions, which it didn’t name, also withheld their votes at the shareholder meeting in Manhattan yesterday.

A spokeswoman for New York Times, Catherine Mathis, didn’t immediately return a call seeking comment.

Shares of New York Times are down 25% in the past year and 45% in the past three years. The company last week reported a 69% drop in net income on costs to eliminate jobs. The stock rose 32 cents to $25.34 in New York Stock Exchange Composite trading.

“We believe that declassifying the share structure will foster accountability,” said Alexander Roepers, president of Atlantic Investment Management, a New York-based hedge fund that owns 2.3% of New York Times shares.

Mr. Roepers, whose firm manages $4.1 billion, didn’t withhold his votes for the company’s slate of directors, though he agrees with Mr. Elmasry’s view on the two classes of stock.

Morgan Stanley conveyed its concerns to the board and senior management on “a number of occasions,” according to the statement. The firm also suggested “substantive strategies to operate the business better,” the statement said.

“However, to date, the board and management have failed to take the actions necessary to improve operational and financial performance,” Morgan Stanley said.

As the company underperformed its peers, management’s compensation has gone up, Morgan Stanley said.

Many newspaper companies have structures that enable their ruling families to maintain control through shares with more voting power. Rupert Murdoch has a separate structure for News Corporation, as does the Bancroft family at Dow Jones and Sumner Redstone for CBS Corporation and Viacom. Cable companies such as Comcast and Cablevision Systems are also run by families who maintain control through a separate class of shares.

Readers Digest Association in 2002 bowed to pressure to give up its two classes of shares. The voting power of Wallace-Reader’s Digest Funds shrank to 14% from 50% in the move.

Knight Ridder, owner of the Miami Herald and Philadelphia Inquirer, sold itself to the McClatchy Company for $4.5 billion to mollify its three largest shareholders.

At the New York Times meeting yesterday, 13 directors were up for election. The Class A nominees, for which Morgan Stanley withheld its votes, were Raul Cesan; a former Federal Communications Commission chairman, William Kennard; a vice chairman of Procter & Gamble, James Kilts, and the chief financial officer of Verizon Communications, Doreen Toben, said.

Arthur Sulzberger Jr., his sister Cathy Sulzberger, and their cousins Michael Golden and Lynn Dolnick are among board members appointed by Class B shareholders.

Morgan Stanley Investment Management is owned by Morgan Stanley, the world’s third-largest securities firm. The investment management firm said it holds stock in the New York Times on behalf of clients, including institutional investors and pension funds around the world.

Shares of Morgan Stanley rose $2.03 to $65.64.

Separately, New York Times boosted its quarterly dividend on Class A and Class B shares to 17.5 cents a share from 16.5 cents. The dividend is payable on June 13 to shareholders of record on June 1, the company said in a statement.