Skip to content

πŸ“‘ Generates a minisearch powered GraphQL API from a directory of markdown files. Additional metadata like tags or descriptions can be added to the files in the form of YAML front matter, which will then be indexed and available to search by.

License

Notifications You must be signed in to change notification settings

TimMikeladze/markdown-to-api

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

28 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Markdown To API

This package generates a GraphQL API from a directory of Markdown files. Additional metadata like tags, descriptions, or custom fields can be added to the Markdown files in the form of YAML front matter, a simple schema at the top of each file. These fields will be indexed and available to query and filter by in the GraphQL API.

Installation

npm install markdown-to-api --save
# or
yarn add markdown-to-api
# or
pnpm add markdown-to-api

Example

This example repository shows how markdown-to-api is used in conjunction with NextJS's Static Generation and Server-side Rendering features to create a searchable site with all the content residing in markdown files. The deployed example can be seen at https://markdown-to-api-example.vercel.app.

Usage

Initialize

import { MarkdownAPI } from 'markdown-to-api'

const mdapi = new MarkdownAPI({
  directory: `./markdown`,
  // In development mode a minisearch index file will be generated.
  writeIndex: process.env.NODE_ENV !== 'production',
  // In production mode, if a minisearch index file is found, it will be used.
  useIndex: process.env.NODE_ENV === 'production',
  // This field is optional, but if provided minisearch will be instantiated with these options. See [minisearch](https://github.com/lucaong/minisearch) to learn more. This is helpful to adjust how your files are indexed and searched.
  miniSearchOptions: {},
});

Content and Configuration

Markdown files

Each markdown file can contain an optional YAML front matter schema.

Below is markdown file showcasing such a schema.

---
title: About Tigers
description: The tiger (Panthera tigris) is the largest living cat species and a member of the genus Panthera.
tags:
- cat
- tiger
genus: Panthera
---

# About Tigers

The tiger (Panthera tigris) is the largest living cat species and a member of the genus Panthera.

It is most recognisable for its dark vertical stripes on orange fur with a white underside. An apex predator, it primarily preys on ungulates, such as deer and wild boar.

The following keys are special in the context of markdown-to-api. They will be indexed and available to query and filter by in the generated API.

  • id: A unique identifier for the markdown file. (Optional, if not provided it will be generated based off of the file path and title.)
  • title: The title of the markdown file. (Optional, will be generated from the filename if not provided.)
  • description: The description of the markdown file. (Optional)
  • tags: An array of tag ids for the markdown file. (Optional). If a config.yml file exists than these tag ids must correspond to the tags defined in config.yml
  • createdAt: An ISO 8601 formatted date string. (Optional)

Config file

In the root of the directory where the markdown files are located, a config.yml file can be used to define tags and other metadata such as making certain fields required.

In the example below we define a list of tags pertaining to felines, make the tags field required and define a new field genus marked as required. Now each markdown file we define must include tags field and a genus field.

tags:
  cat:
    description: Fits into any box.
  puma:
    name: cougar / puma
    description: Puma is a large cat, also known as a mountain lion.
  tiger:
    description: Tiger is a very large cat.
  lion:
    description: Lions have manes.
fields:
  tags:
    required: true
  genus:
    required: true

CLI

markdown-to-api generates an index.json file used for searching through the markdown files. During development this file will be generated automatically, however when running via CI/CD or as part of your build process it is useful to generate this file via a script.

node node_modules/markdown-to-api/dist/cli.mjs -d markdown

This command can be added to the scripts section of your package.json file. See the example repository to see how this works in action.

Searching

// Search all files for the term `cat`.
mdapi.getIndex().search('tiger');

// Search only tags field for the term `tiger`.
mdapi.getIndex().search('tiger', { fields: ['tags'] });

Additional options can be passed to the search method. This is an options object which is passed to minisearch.

GraphQL API

GraphQL Module

A GraphQL Module is included in this package which encapsulates the schema and resolvers provided by markdown-to-api. The example below shows how you can instantiate a new module which can then be consumed by your GraphQL server of choice.

import { createApplication } from 'graphql-modules';
import createMarkdownAPIModule } from 'markdown-to-api';

const markdownAPIModule = createMarkdownAPIModule({
  directory: `./markdown`,
  // In development mode a minisearch index file will be generated.
  writeIndex: process.env.NODE_ENV !== 'production',
  // In production mode, if a minisearch index file is found, it will be used.
  useIndex: process.env.NODE_ENV === 'production',
});

const app = createApplication({
  modules: [markdownAPIModule],
});

Example queries

query SearchMarkdownFiles {
  searchMarkdownFiles(text: "tigers") {
    count
    results {
      id
      title
      description
      createdAt
      tags {
        id
        name
        description
      }
      markdownFile {
        content
      }
    }
  }
}
query SearchTags {
  searchMarkdownFiles(text: "lion", options: { fields: ["tags"] }) {
    count
    results {
      id
      title
      description
      createdAt
      tags {
        id
        name
        description
      }
      markdownFile {
         content
      }
    }
  }
}
query AllFilesAndTags {
  countMarkdownFiles
  markdownFiles {
    id
    title
    content
    tags {
      id
      name
      description
    }
  }
  markdownFileTags {
    id
    name
    description
  }
}

Generate schema

type Query {
  countMarkdownFiles: Int
  markdownFile(id: ID!): MarkdownFile!
  markdownFiles(direction: MarkdownFilesSortDirection, limit: Int, offset: Int): [MarkdownFile!]!
  searchMarkdownFiles(text: String!, options: MarkdownFileSearchOptions, limit: Int, offset: Int): MarkdownFileSearchResults!
  autoSuggestMarkdownFileSearch(text: String!, options: MarkdownFileSearchOptions): MarkdownFileAutoSuggestResults!
  markdownFileTags: [MarkdownFileTag!]!
}

type MarkdownFile {
  content: String!
  strippedContent: String!
  id: ID!
  path: String!
  slug: String!
  tags: [MarkdownFileTag!]!
  title: String!
  description: String
  createdAt: String!
}

type MarkdownFileSearchResults {
  count: Int!
  results: [MarkdownFileSearchResult!]!
}

type MarkdownFileSearchResult {
  id: ID!
  description: String
  score: Float!
  terms: [String!]
  title: String!
  slug: String!
  path: String!
  tags: [MarkdownFileTag!]!
  createdAt: String!
  markdownFile: MarkdownFile!
}

type MarkdownFileTag {
  id: ID!
  name: String!
  description: String
}

input MarkdownFileSearchOptions {
  fields: [String!]
  weights: MarkdownFileSearchOptionsWeights
  prefix: Boolean
  fuzzy: Boolean
  maxFuzzy: Int
  combineWith: MarkdownFileSearchOptionsCombineWith
}

type MarkdownFileAutoSuggestResults {
  count: Int!
  results: [MarkdownFileAutoSuggestResult!]!
}

type MarkdownFileAutoSuggestResult {
  suggestion: String!
  terms: [String!]!
  score: Float!
}

enum MarkdownFileSearchOptionsCombineWith {
  AND
  OR
}

input MarkdownFileSearchOptionsWeights {
  fuzzy: Float!
  exact: Float!
}

enum MarkdownFilesSortDirection {
  asc
  desc
}

enum MarkdownFilesSortBy {
  title
  createdAt
}

About

πŸ“‘ Generates a minisearch powered GraphQL API from a directory of markdown files. Additional metadata like tags or descriptions can be added to the files in the form of YAML front matter, which will then be indexed and available to search by.

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published