Methodology

Overview

The Library is a centralized repository of information on the learn-and-work ecosystem. The information  is of three types: 

Publicly available information. Examples include descriptions of initiatives and organizations that are working to improve the learn-and-work ecosystem.

Value-added content—that is, resources developed by our Library Team, which includes  commissioned researchers and writers. Examples of value-added content include articles on topics in the ecosystem that are informed by literature review and survey research used to develop the ontology for the Key Components of the ecosystem.

Wiki-model content—that is, information obtained through an open-access process for submissions and edits provided by our community of users. Wiki-model content is brought to the attention of the library through a submitted form.

 

Data Categories and Metadata Tagging

Every item entered into the Library is coded by the Library Team according to various categories. This coding system organizes the information into several searchable datasets.

Primary Categories

TypeDescribesCriteria
Key ComponentsThe 12 crucial elements of the learn-and-work ecosystem, facilitating seamless individual navigation and informed public engagementdefined_by == “key-components”
Stakeholder groupsThe diverse community of stakeholders working across the 12 areas of the ecosystemdefined_by == “stakeholders”

Entity Types

SubtypeDescribesCriteria
Archive ItemsHistorical documents, including languishing or disappearing websites, important for research stakeholdersentity_type == “archive-items”
GlossaryConcise definitions of terms and concepts crucial to the learn-and-work ecosystementity_type == “glossary”
InitiativesKey projects working to improve the learn-and-work ecosystementity_type == “initiative”
OrganizationsAlliances and intermediary organizations focused on various areas of the learn-and-work ecosystementity_type == “organization”
TopicsImportant subjects or themes defining the learn-and-work ecosystementity_type == “topic”

Tips for Searching at the Library

The Library is built to facilitate multiple types of searches. Some users will come to the Library knowing what topic(s) they are looking for; others will be unsure what questions to ask and simply wish to understand the learn-and-work ecosystem. To serve all types of users, the Library offers several ways to search for information—all consistent with general searching techniques of other databases and libraries.

  • The Main Page is designed to depict the major types of information available. You will see categories in tiles: In one tile, Glossary Terms, Index, Archive and Key Components. In another tile, you will see Initiatives, Organizations, and Topics.
  • A general Search Bar in the top menu allows a user seeking specific information to type in that query. The search will return a comprehensive list of results that includes relevant content from all of the Library’s seven datasets     
  • On the right, you will find two tabs for Index and Glossary. The Index lists all content at the Library in alphabetical order. Each type of content is color-coded to make it easier for you to know what type of content is available, especially where there are duplicate entries.  As example, there is a Glossary entry for Accreditation and also a Topic (an article) for Accreditation.
  • Search Filtering Options. This feature enables users to specify which types of content they seek (Glossary Term, Component, Topic, Initiative, Organization), and identify which type of Stakeholder they are. A Search Bar is provided for this filtering feature.     
  • Live links are provided throughout the Library to give users quick access to related external definitions or relevant sites on the Internet.
  • Information included in the Library Archive is available on request and is not fully searchable.  Please contact the Librarian if you wish to search archived resources, as this feature remains under development.

Accuracy and Updating

  • To ensure fact-based content, the Library team reviews publicly available information about Initiatives and organizations and removes subjective language. Claims that describe a program as “the only one,” the “leading effort,” or a “bold effort” are untested and are viewed as marketing tools. We remove them before items are entered into the Library.The Library team brings human judgment to the collection and updating process. We regularly read several trade publications to identify content, including social media.
  • The Library is a wiki model: there are easily accessible forms at the Library to enable users to submit requests for changes, which we will review; and to request new entries.
  • The Library is continually developing methods to keep updated content. In 2024, we will be inserting dates and the funder(s) for initiatives included in the Library to make it easier to search for items and place them in a proper time context.
  • We are working to add findings and outcomes from initiatives when they become publicly available, subject to review by the Library team for fact-based evidence.
  • Documents are reviewed on a regular basis to determine if there is updated information about items in the Library.
  • The Library is exploring the use of artificial intelligence to augment our human judgment in the updating process. AI assisted relational maps are developed by integrating manual data tagging with inferred AI-driven relations. This work is performed under an open licensing agreement that allows the Library to train and continuously refine the AI model.

Limitations of the Library Collection

  • The Library is not a complete repository of information describing the learn-and-work ecosystem.  It is an evolving collection that began to gather information related to the 12 key components (the ontology for the Library) selected for the Library as a result of research conducted in 2019. Research which attempts to portray the collection as a balanced view of Initiatives working to improve the ecosystem or the Organizations working to improve the ecosystem would not be accurate if the Library was the only source of information.     
  • The Library is premised on referring users to original sources, primarily websites or reports. Websites maintained by organizations that describe work in the ecosystem are often limited: Some have no search function, some have limited information or dated information. Users may quickly find themselves dead-ended when they visit such websites.     
  • Content from international sources has been edited to conform to American English. Spellings for international content will likely differ from original documents the Library team pulled in.
  • Relational mapping is a new way the Library is exploring to depict the relationships among components of the ecosystem. The maps have been developed by combining AI searching and human judgment. Some categories of information will be more relevant to the main search item than others. The Library team is working to develop a way to “weight” items included in the maps. These will still be subject to human judgment, so there will be an imperfect alliance in depicting  relationships. Maps will show a selection of ecosystem connections, but can’t be seen as comprehensive lists of related content.
  • Synthesis of information, depicting what the information means, is another major challenge—one that the Library seeks to address through its planned Research Center. The Library team will be searching its information database and developing trend reports, short briefs, and other syntheses of information on activities and trends in the ecosystem.  Researchers are invited to contact the Librarian to conduct research and determine how or if the researcher might affiliate with the Research Center.

Organizations (274)

Initiatives (294)

Topics (93)