The Obama administration has unveiled its plan for measuring how well the stimulus package works, and the bottom line is: Many questions will remain unanswered.

While pitching the stimulus package, the administration batted around the number 3.5 million when talking up how many jobs would be created or saved, based on an estimate by Christina Romer, who now heads the Council of Economic Advisers, and a colleague in what was then the office of Vice President-elect Joe Biden.

The administration’s jobs estimate relied on some broad rules of thumb (otherwise known as multipliers), assuming, for example, that every 1 percent of GDP the government spends will result in a 1.6 percent increase in GDP and that every 1 percent increase in GDP equals about 1 million jobs saved or created.

But that was just an estimate to pitch the plan, and soon hard numbers will start flooding in from agencies and contractors telling us exactly how many workers the $787 billion stimulus package has saved, right?

Not exactly.

A report recently released by the Council of Economic Advisers (PDF), a three-member team advising the president on economic affairs, suggests that real-world data will not supply a firm standard of accountability for stimulus spending any time soon. The report is the first in a quarterly series tracking the effects of the stimulus.

In the short run, the council instructs agencies to use the rule that $92,000 equals one job-year (one job for one year) to determine how much employment their programs are generating. The $92,000 number is derived using the administration’s earlier macroeconomic assumptions and, the report says, is intended to make agency estimates mirror the administration’s. This will make the agencies’ numbers a poor check on the administration’s job figures.

Later, data will begin to come in from recipients of stimulus funding, which are required to carefully report how many jobs are created or saved. This is more complicated than it seems. Surely you count the employees of the construction contractor hired to renovate the government building, for example, but do you count the subcontractors hired by the contractor? And what about workers at the paint factory who would have been laid off without business from the construction project? What about workers at the nearby grocery store who keep their jobs because of increased business from hungry construction workers and paint manufacturers?

The report offers a sneak peek at the final guidelines the Office of Management and Budget is expected to release soon for addressing these very questions.

In the renovation example above, the construction contractor’s employees and the subcontracted workers would count as jobs created or saved, but the other jobs wouldn’t, according to the guidelines.

The contractor would then submit a report with the number of jobs created and saved, expressed in full-time equivalents (the total hours of work created divided by the number of hours worked by a full-time worker), along with a brief description of the types of work involved, which will later be used to evaluate the quality of jobs created by stimulus funding.

Lest employers be tempted to cheat, the OMB will check the data for “completeness and plausibility” by looking for outliers among similar projects and cross-checking information about expenses and wages with labor market data from the region.

Why can’t we just add up the numbers from these reports to find an aggregate number of jobs created by stimulus spending? There are several problems with this, according to the council.

The first is that those reports will only document job creation from the $271 billion in direct government spending. The rest of the stimulus was spent on tax cuts, state fiscal relief and transfer payments like Social Security and food stamps; no direct reports will ever come in on the number of jobs created by those expenditures.

The second is that the reports will only measure direct job creation (think of the construction example above) and not indirect effects, even though the administration is counting on indirect job creation to reach its 3.5 million job mark at the end of 2010.

Finally, the council apparently doesn’t trust those reports very much.

“There will likely be inconsistencies and measurement error across the individual reports,” the council writes. “This limitation is present whenever thousands of recipients with very different types of projects are asked to provide information.”

But the Obama administration does have a rough estimate of how many jobs will be created by direct government spending. (For the record, $100 billion creates 1,085,355 job-years.) So what happens if its estimate differs from the direct job-creation reports?

The answer, given by a senior administration official, is enough to make any social scientist squeamish: “It will be a two-way test of how good the numbers we get back will be and also a test of multipliers.”

In other words, if the job-creation numbers the administration gets from real-world data disagree with its estimates, they reserve the right to blame the data. From an accountability perspective, this will make it difficult to assess the stimulus’ successes and failures.

But Romer and her associates propose some avenues for evaluating their own estimates, like regular checks to see if money has gone out on schedule, microeconomic analyses to estimate the number of indirect jobs that result from government spending, and comparisons of the unemployment rate with baseline forecasts of the unemployment rate without stimulus.