What’s New

This page is updated monthly with a list of new ([new]) and improved ([improved]) features and bug fixes ([fixed]) to our website, API, data pipelines, and data holdings. We also list operations issues and improvements of note ([ops]).

To receive this listing via monthly email newsletter, along with announcements and new blog posts, sign up here.

October 2019

  • Website and API
    • [new] Basic, HTML version of ADS now available; links are shown in the splash screen and footer
    • [new]A Final Farewell to ADS Classic,” a blog post with more information about the retirement of Classic
    • [new] Added information on data curation and availability, formerly available in ADS Classic, to a new Legacy menu in the help pages
    • [new] Added handling for myADS queries linked in Classic-generated myADS emails
    • [improved] Updates to the Paper Form, including addition of the reference resolver and styling updates
    • [improved] Updates to the Classic Form, including addition of a sort selection and a General collection checkbox; collection checkboxes now respect user settings selections, have updated wording
    • [improved] Library updates: deleting an item from a library no longer takes you back to the first page of the library
    • [improved] Added authorcutoff option for BibTeX export, to specify the max number of authors to show
    • [improved] Added search syntax for searching for affiliations to help pages
    • [improved] Added Github data source
    • [improved] For redirected links from Classic, all warnings about using AND vs OR have been removed; ORs are no longer converted to ANDs
    • [fixed] Fixes to Classic Form keyword parsing
    • [fixed] Fixed normalized citation count on results page
    • [fixed] Fixed bug that caused some Library Link URLs to not work properly
    • [fixed] Export service: fixed ADS fielded format to return ISSN instead of ISBN
    • [ops] Update website to use jQuery to v2.2.4, added Reactify plugin
    • [ops] Converted bigquery to use docs
    • [ops] Internal improvements to the resolver gateway and resolver microservice
  • Data holdings and pipelines
    • 37,583 more records, 605,305 more citations
    • [ops] Our VMWare-based pipeline servers had to be restarted.
    • [ops] Over the last week, 99.93% of queries returned successfully.
Development details

September 2019

  • Website and API
    • [new] Added option to transfer ownership of a library, accessible from the Manage Access tab in the library
    • [new] In Application Settings, added the ability to set a default search form (Modern, Classic, Paper)
    • [new] Added infrastructure endpoint to eventually support myADS settings imports
    • [improved] If sidebars are hidden, requests to populate them are now blocked to improve load speed
    • [improved] Library exports are now using the docs() syntax, to improve speed
    • [improved] Added more flexibility for specifying BibTeX key format and max author settings for BibTeX and BibTeX ABS formats
    • [improved] Improved scoring algorithm deployed; sort by score to use
    • [fixed] Bug fixes to web interface: Classic search form now respects collections preferences set in Application Settings, fixed bug that caused the number of results to occasionally disappear on the results page, fixed bug for displaying special characters in the publication field on the abstract page
    • [fixed] Fixed which field is being exported as the publisher field
    • [ops] Add more info to logging requests
    • [ops] Streamlined and added to Google Analytics timing logging
    • [ops] Internal updates to builds and dependencies
  • Data holdings and pipelines
    • 53,878 more records, 613,711 more citations
    • [fixed] Fixed citation_count field, some records had no value instead of a 0
    • [fixed] Reviewed and fixed metadata parse issues for all bibcodes
    • [fixed] Fixed publication data on PNAS records
    • [fixed] Fixed 2019GeoJI records.
    • [ops] Updated affiliation field for all bibcodes
    • [ops] Over the last week, 99.99% of queries returned successfully.
    • [ops] Fulltext pipeline XML extraction updated to BeautifulSoup4
Development details

August 2019

  • Website and API
    • [new] Activated symbolic date math logic. Searches like entdate:[“NOW-1MONTH” TO NOW] will now work
    • [new] Videos section added to help pages, including recordings of Office Hours talks
    • [improved] Several improvements to Solr search engine scoring: improved the algorithm for choosing amongst synonyms, updated handling of multi-phrase synonyms, improved relevancy scoring, allow for multi-multi-valued affiliations (e.g. authors with multiple affiliations), improved handling of author accents with transliterations
    • [improved] Added clarification to Help pages’ FAQ section about replacements for Classic’s “Get citation lists” and “Get reference lists” functions
    • [fixed] Bug fixes to export service: better detection of math mode, limit the number of authors in MNRAS-style export
    • [fixed] Fixed bug when there is a caret in the query string in object searches
    • [fixed] Fixes to Solr search engine scoring: fixed NGC object search handling/tokenization, fixed year:xxxx-yyyy searches, fixed issue with constant scoring for position searches, fixed bugs affecting authors with multiple initials, scoring fix for first-author search using the caret (^) operator
    • [ops] More info included with feedback form emails
    • [ops] Better internal logging, including logging user ID with requests
    • [ops] Better error handling for failed cone object searches
    • [ops] Improved Solr warm-up, for performance
  • Data holdings and pipelines
    • 47,005 more records, 835,256 more citations
    • [fixed] Fixed bug in search code that affected 0.5% of facet searches. Now, 99.97% of queries return now successfully.
    • [fixed] The fulltext pipeline had inconsistent encoding issues that were fixed by detecting the encoding of a file and decoding it before parsing.
    • [ops] Daily updates failed on two occasions, one due to central server room network issues and one from an abnormally large log file filling a disk.
    • [ops] Updated 24469 ICRC records, 4309 arXiv records, 1114 Zenodo records, and 25 Minor Planet Bulletin records.
    • [ops] Improved handling and internal reporting of fulltext pipeline errors.
Development details

July 2019

  • Website and API
    • [new] Citation key in BibTeX exports is now configurable
    • [new] Maximum number of authors to display in a BibTeX export is now configurable
    • [new] Documentation: object search syntax added to search syntax help pages, new tutorial available for generating files for UI troubleshooting
    • [new] New blog post on user-developed tools
    • [improved] Updates to feedback form, including added links to correction forms
    • [improved] Application settings page now auto-saves
    • [improved] Documentation: overhaul of custom export format documentation
    • [fixed] Fixed issue for Internet Explorer 11 users
    • [fixed] Updated BibTeX export files to use .bib instead of .bbl extension
    • [fixed] Fixed export menu ordering
    • [fixed] Fixed table of contents bugs
    • [fixed] Fixed library sorting bug
    • [fixed] Fixed property logic in Classic form
    • [fixed] Fixed issue with handling long author list in exports
    • [fixed] Fixed bug for records with a certain EID format that occurred with some export formats
    • [fixed] Updates to date/pubdate and bibstem fields in exports
    • [fixed] Fixed AASTeX export format issue
    • [fixed] Fixed indentation on custom export formats
    • [fixed] No longer doing TeX manipulation of strings in math mode during export
    • [fixed] Fixed several logging error messages
    • [fixed] Fixed cone search functionality in object searches
    • [ops] Now using https instead of http for calls to NED and SIMBAD
    • [ops] Updated rate limiting for searches
  • Data holdings and pipelines
    • 45,499 more records, 582,817 more citations
    • [ops] References were missing for at least 75 bibcodes. This was corrected by resending all reference data.
    • [ops] Detected at least two bibcodes are missing the arxiv_class field. Correction to metadata has been requested.
    • [ops] Updated our affiliation database with hundreds of new definitions and reprocessed all affiliation strings.
Development details

June 2019

  • Website and API
    • [new] Select desired articles and limit search results to the selected set using the new “limit to” button (visible after selecting one or more articles)
    • [new] Search examples on Modern form are now clickable links
    • [improved] Improved wording and functionality when using second-order operators from the Explore menu for a selected group of articles
    • [improved] About menu includes link to What’s New page
    • [fixed] Better error handling and improved performance issues
    • [fixed] Improved client-side caching
    • [fixed] Fixed sorting by normalized citations
    • [fixed] Fixed sorting during export
    • [fixed] Fixed issue to now allow results to open in new tab
    • [ops] Improved session handling
    • [ops] Increased rate limits for legitimate traffic
  • Data holdings and pipelines
    • 54,835 more records, 767,576 more citations
    • [improved] Improved reliability of data ingest pipeline
    • [improved] Updated solr records to use improved bibstem and institution fields
    • [fixed] Changed default value for normalized citation count field to fix sorting issue
    • [ops] Fixed publisher bibcode for a set of conference papers
    • [ops] Deployed support for author/reviewer attributes
    • [ops] Recovered from server room cooling failure
    • [ops] Fixed links data for workshop
Development details

May 2019

  • Website and API
    • [new] Second order operators now available for a manually selected group of articles via the Explore menu
    • [improved] Library set operations - improved: user experience, error handling, default library descriptions, tool tips
    • [improved] Improved usage of reCAPTCHA
    • [improved] Removed day from pubdate quick-field format
    • [improved] Added IRSA data provider
    • [improved] Updated classic form to allow hyphenated search terms
    • [improved] Detect ctrl+click on links to help with Windows users
    • [improved] Added warning messages in the Classic Search Translator for title and abstract search errors
    • [fixed] Fixed routing issue that caused 404 pages to trap users
    • [fixed] Fixed issue causing stuttered scrolling on iOS devices
    • [fixed] Fixed associated papers widget to show properly
    • [fixed] Quote author names by default in classic form
    • [fixed] Fixed export output to remove extraneous characters
    • [fixed] Fixed public library routing Issue, and update other links with same issue
    • [fixed] Fixed library export control to respect selected sort
    • [fixed] Fixed issue causing abstract citation list to have wrong title
    • [fixed] Removed synonym replacement for object searches
    • [ops] Added content-type for bigquery requests from several internal services
  • Data holdings and pipelines
    • 61,016 more bibcodes, 719,093 more citations
    • [improved] Overnight arXiv records are now available sooner, due to code improvements
    • [improved] Improved error handling for OCR errors in papers
    • [improved] Improved generation of bibcodes when the page number overflows its field
    • [improved] Improved quality of metrics data
    • [new] In development: code to generate pages for web crawlers
Development details

April 2019

  • Website and API
    • [new] Implemented lazy loading of widgets to improve site performance
    • [new] New feature: library set operations, available under Action button within library module
    • [new] New search field: “inst” (institution) field is now available for users to search the set of curated affiliations, currently available from the left-side filters; this complements the existing “aff” (affiliation) field, which allows users to search the uncurated text of this field
    • [improved] Additional export formats added to be more compatible with Classic-style export
    • [improved] arXiv ID is now pushed with ORCID claims to improve record matching on the ORCID site
    • [improved] Improvements to showing/hiding the sidebars
    • [improved] Improved UX on fast connections
    • [improved] arXiv ID added to abstract screens
    • [fixed] Fix for default naming in library set operations
    • [fixed] Fixed several bugs for exporting citations: fixed G author format, fixed journal name for macro matching, fixes to be more compatible with Classic-style export
    • [fixed] Fixed several bugs in Classic query translator: fixed bugs in parsing bibstems to include/exclude, fixed bug when querying for a single author, fixed bug to ensure database selection is passed, cleaned up error messages
    • [fixed] Fixed bug to speed site load
    • [fixed] Fixed several ORCID bugs related to the ORCID login, claiming records with no abstract, claiming/updating records using an alternate bibcode, updating records from the ORCID profile screen, requiring user to log into ORCID again if ORCID login has expired
    • [fixed] Fixes to the Classic and Paper forms
    • [fixed] Fixes to library interface including who could view public libraries, fix for viewing large libraries in search results screen, fix for adding record to a library from the abstract screen
    • [fixed] Numerous bug fixes in the user interface to improve user experience
    • [fixed] Fixes to link handling and routing
    • [fixed] Fixes to ensure compatibility with older browsers and for users who have Google blocked
    • [ops] Fix to internal user account cleanup script
    • [ops] Infrastructure to support forthcoming new feature: docs() operator, for performing operations on a selected group of articles or library
    • [ops] Initial deployment of service to load static pages for abstracts to improve site speed; service still in development
    • [ops] Internal build-related updates, updates to continuous integration, updates to unit testing infrastructure
    • [ops] Changes in how errors from the Classic query translator are handled
  • Data holdings and pipelines
    • 51,795 more bibcodes, 799,237 more citations
    • [improved] Complete records from arXiv papers are now searchable several hours earlier.
    • [ops] Performance of our search engines continues to be monitored and improved.
Development details

March 2019

  • Website and API
    • [fixed] Fixed error for users with no works on their ORCID profiles
    • [operations] Updated “CXO” to “Chandra” as a data source name
    • [operations] Updated expiration dates for access tokens
  • Data holdings and pipelines
    • 57,929 more bibcodes, 836,963 more citations
    • [new] The augment affiliation pipeline was productized and has reprocessed millions of affiliation records
    • [improved] The full text pipeline was improved to better handle text in appendices and text from specific publishers
    • [improved] Production search engine performance has been improved
    • [operations] We had some issues with a Solr search engine instance restarting; we are still investigating
Development details

February 2019

  • Website and API
    • [improved] Better handling of requests to API to improve speed
    • [improved] Major back-end ORCID update, including many speed improvements, particularly for users with large profiles
    • [improved] Added new CfA logo to footer, updated footer styling
    • [improved] “Export” renamed to “Export Citation” on abstract page
    • [improved] Added “entdate” field and “similar” operator to Quick Fields menu
    • [fixed] Fixed issue where default database/collection was not always applied, even if set in preferences
    • [fixed] Fixed handling if an ORCID login fails
    • [fixed] Operators that require sorting by score are now sorting properly
    • [fixed] Copying and pasting into the search box no longer triggers the autocomplete
    • [fixed] Fixed export of accented characters
    • [fixed] Fixed behavior of %U, which outputs the URL of the bibcode, in custom export
    • [fixed] Year and month in BibTeX export are now wrapped in double quotes, per the BibTeX standard
    • [fixed] Fixed line breaks in the author and editor fields in the BibTeX export to not wrap in the middle of a name
  • Data holdings and pipelines
    • 65,903 more bibcodes, 769,972 more citations
    • [improved] We deployed improvements to our orcid pipeline and our master pipeline.
    • [improved] We continued to productize the affiliation enhancement pipeline.
    • [operations] Zendo based software citations ran in production mode.
    • [operations] All pipelines ran without interruptions for the month.
Development details

January 2019

  • Website and API
    • [new] Set operations (union, intersection, difference, copy) available for libraries when using the API (not yet available in the browser).
    • [fixed] Fixed how the custom export handles user-supplied LaTeX commands.
    • [operations] Internal improvements to request handling.
  • Data holdings and pipelines
    • 32,886 more bibcodes, 527,614 more citations
    • [new] Results of our affiliation enhancement project are becoming visible to users. We deployed improved affiliation data and faceted search support for several million bibcodes.
    • [new] Our first software citations via Zenodo (bibstem:zndo) are available.
    • [operations] Ingest pipelines was down for a day due to server room issues.
Development details