For the basics of how the federal budget works, the authoritative source is the section on "Budget Concepts and Budget Process" from OMB's annual Analytical Perspectives. Anyone with an interest in the budget that extends much beyond what I present here should start with at least a quick look at it.
The Office of Management and Budget (OMB) is the official keeper of the federal budget. Its publications contain arguments in favor of the administration's proposals and policies, but also huge amounts of purely factual information, for which in many cases OMB is the ultimate authority.
The Congressional Budget Office (CBO) is a highly professional and studiedly neutral supporting organization for Congress. I keep holding my breath waiting for CBO to be done away with for failing to provide cooked "facts" to support the policies of powerful people in Congress — it has come under a lot of pressure from time to time and other offices have suffered such fates. But in the meantime it remains an excellent source of information and analysis.
The Bureau of Economic Analysis (BEA) of the Dept. of Commerce is not budge/tax oriented, but it is the master source of information and statistics about the economy as a whole. It is the keeper of the National Income and Product Accounts (NIPA), and heir to the tradition of Simon Kuznets in national income accounting. Among other things the NIPAs coherently account for government in the context of the nation's economy, presenting a different picture than that portrayed in the more administratively (and politically) oriented budget.
The Organisation for Economic Cooperation and Development (OECD) is the rich nations' club and has good statistical resources, a considerable portion of which it makes freely and readily available to the public.
Executive departments and agencies often have their own budget sites with more detail than OMB offers. The Social Security Board of Trustees publishes information about the finances of Social Security and the Centers for Medicare and Medicaid Services provides comparable information for their part.
Here are some links to organizations and groups that also offer information about budget and related issues. Except as I caution, so far as I'm aware they all present objective and undoctored data, but you have to judge for yourself regarding their policy views. I've left out a lot of places that offer some incidental information but are primarily focused elsewhere.
Cato Institute Downsizinggovernment.org. Quirky and something of a mixed bag so far as the data goes.
Center on Budget and Policy Priorities
Heritage Foundation. Not my own personal choice of a source for information on anything, but make up your own mind.
Kaiser Family Foundation. Much on health policy.
Tax Policy Center of the Urban Institute and Brookings Institution