Skip to content
Joshua Tauberer edited this page Dec 4, 2016 · 5 revisions

Running the Scraper

Amendments are scraped as a part of the bill scraper. The bill scraper outputs amendment files for any bill it processes. (As of July 2016 there is no longer a separate amendments scraper. Amendments on nominations are treaties are no longer scraped as the only source of structured data on amendments is via bills.)

Output Data Format

Every amendment has a JSON file, data.json, with fields related to a amendments's ID, status, and history. There is also a corresponding XML file, data.xml, which is roughly compatible with GovTrack's legacy data format; however, the XML format is not documented, and is not particularly recommended.

The output files are in the following location:

data/[congress]/amendments/[amendment_type]/[amendment_type][number]/data.{json.xml}

See the bills documentation for a definition of congress, and see below for the definitions of amendment_type and number.

The data examples below use excerpts from H.Amdt. 183 in the 113th Congress where possible.

Basic Information

{
  "amendment_id": "hamdt183-113",
  "amendment_type": "hamdt",
  "chamber": "h",
  "congress": "113",
  "number": 183,
  "updated_at": "2013-11-02T07:31:09-04:00"
}

Bills are uniquely identified by a Congress, amendment type, and number, much like bills. Amendments die at the end of a Congress and numbering resets with each Congress. (See bills.)

amendment_type and chamber indicate the type of the amendment. In almost all cases amendment_type is either hamdt for House amendments or samdt for Senate amendments. In the 97th and 98th Congresses, there appear some "Senate Unprinted Amendments". For these amendments, the amendment_type is supamdt. The chamber property is either h (for hamdt) or s (for samdt and supamdt).

Amendment IDs are in the form of [amendment_type][number]-[congress].

updated_at is the date and time that the JSON file was last saved. It reflects the time the scraper was run and is not metadata about the amendment itself.

What the amendment amends

An amendment can amend a bill, a treaty, or even another amendment.

Here is an example of the most common case:

{
  "amends_amendment": null,
  "amends_bill": {
    "bill_id": "hr1947-113",
    "bill_type": "hr",
    "congress": 113,
    "number": 1947
  },
  "amends_treaty": null
}

When the amendment amends a bill, you will see the amends_bill property set and the amends_amendment and amends_treaty properties set to null. These fields correspond with the same fields in bills.

When an amendment amends another amendment, the amends_amendment property is set to the other amendment's basic metadata. In addition, the amends_bill or amends_treaty (see below) property is set to the bill or treaty that is eventually amendment (it may be an amendment to an amendment to an amendment to a bill). Here is an example from H.Amdt. 10 in the 113th Congress:

{
  "amends_amendment": {
    "amendment_id": "hamdt5-113",
    "amendment_type": "hamdt",
    "congress": 113,
    "number": 5
  },
  "amends_bill": {
    "bill_id": "hr152-113",
    "bill_type": "hr",
    "congress": 113,
    "number": 152
  },
  "amends_treaty": null
}

Finally, an amendment can also be proposed to amend a treaty, in which case amends_bill will be null and amends_treaty will include which treaty is being amended. Here is an example from S.Amdt. 3 in the 1000th Congress.

{
  "amends_amendment": null,
  "amends_bill": null,
  "amends_treaty": {
    "congress": 94,
    "number": 25,
    "treaty_id": "treaty25-94"
  }
}

It may be possible for there to be an amendment to an amendment to a treaty, in which case amends_amendment would not be null.

Description and purpose

{
  "description": "Amendment clarifies that Corps of Engineers construction projects receiving funds under this legislation will be at full Federal expense only with respect to the funds...",
  "purpose": "An amendment numbered 6 printed in Part C of House Report 113-1 to clarify that the Corps of Engineers construction projects receiving funds in this legislation will be at..."
}

An amendment may have a description or purpose, both, or neither. These fields are copied over from similarly named fields on THOMAS. We do not know the semantic difference between the fields.

Sponsor

{
  "sponsor": {
    "bioguide_id": "W000437", 
    "district": null, 
    "name": "Wicker, Roger F.", 
    "state": "MS", 
    "title": "Sen", 
    "type": "person"
  }
}

The sponsor of the amendment, if there is a sponsor, is set like in the example above. The bioguide_id should be used to cross-reference with the congress-legislators YAML database. Historical data from GovTrack uses thomas_id in place of bioguide_id. See bills for more.

Introduction and status

{
  "introduced_at": "2013-04-24",
  "proposed_at": "2013-04-24",

  "status": "pass",
  "status_at": "2013-05-06",
}

Every amendment has an introduced_at date. The proper terminology seems to be that amendments are "offered" in the House and "submitted" in the Senate. We've normalized this difference to just introduced_at for consistency with bills.

In the Senate, amendments also have a proposed date which is indicated with proposed_at. We don't know what this means. This property is only set on Senate amendments.

status can be one of offered (meaning introduced), pass (amendment was voted on and was accepted), fail (amendment was voted on and failed), or withdrawn (amendment was withdrawn). Our withdrawn detection needs some work. status_at is the date or datetime timestamp when the status changed.

Actions

{
  "actions": [
    {
      "acted_at": "2013-06-19T16:14:00-04:00",
      "references": [
        {
          "reference": "CR H3862-3863",
          "type": "consideration"
        },
        {
          "reference": "CR H3862",
          "type": "text"
        }
      ],
      "text": "Amendment (A008) offered by Mr. Graves (GA).",
      "type": "action"
    },
    {
      "acted_at": "2013-06-19T16:19:00-04:00",
      "references": [
        {
          "reference": "CR H3863",
          "type": "consideration"
        }
      ],
      "text": "By unanimous consent, the Graves (GA) amendment was withdrawn.",
      "type": "action"
    }
  ]
}

Amendments have actions in the same format as bills.

The type of an action can be either vote or withdrawn. When type is vote, the result, how, and roll (for roll call votes) properties will be set and have the same meaning as in bills.

Other numbering

There are at least five ways that amendments are numbered. See the thread at https://github.com/unitedstates/congress/issues/68 for some discussion. In general, the other numbering systems are not reliable because they are ambiguous. You will find the house_number field set on amendments starting with the 101st Congress. This is the A001-style numbering that you'll see on THOMAS. It is assigned consecutively in real time as amendments are offered on the House floor and the number is relative to a particular bill. Although the number appears on THOMAS, it is not found in any other data source reliably. So don't use it.

Clone this wiki locally