Skip to content

Commit

Permalink
Update script to generate Valkey Module API docs (#406)
Browse files Browse the repository at this point in the history
The output of this script becomes the contents of
`topics/module-api-ref.md` in the `valkey-doc` repo. (Updating it is a
manual process.)

The script uses git tags to find the version that first added an API
function. To preserve the history from old Redis OSS versions, for which
we don't keep git tags, a mapping is stored in a file.

Signed-off-by: Viktor Söderqvist <viktor.soderqvist@est.tech>
  • Loading branch information
zuiderkwast committed Apr 30, 2024
1 parent 05251c5 commit 6e05d0f
Show file tree
Hide file tree
Showing 2 changed files with 371 additions and 19 deletions.
36 changes: 17 additions & 19 deletions utils/generate-module-api-doc.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ def markdown(s)
newlines = []
# Fix some markdown
lines.each{|l|
# Rewrite RM_Xyz() to RedisModule_Xyz().
l = l.gsub(/(?<![A-Z_])RM_(?=[A-Z])/, 'RedisModule_')
# Rewrite VM_Xyz() to ValkeyModule_Xyz().
l = l.gsub(/(?<![A-Z_])VM_(?=[A-Z])/, 'ValkeyModule_')
# Fix more markdown, except in code blocks indented by 4 spaces, which we
# don't want to mess with.
if not l.start_with?(' ')
# Add backquotes around RedisModule functions and type where missing.
l = l.gsub(/(?<!`)RedisModule[A-z]+(?:\*?\(\))?/){|x| "`#{x}`"}
# Add backquotes around ValkeyModule functions and type where missing.
l = l.gsub(/(?<!`)ValkeyModule[A-z]+(?:\*?\(\))?/){|x| "`#{x}`"}
# Add backquotes around c functions like malloc() where missing.
l = l.gsub(/(?<![`A-z.])[a-z_]+\(\)/, '`\0`')
# Add backquotes around macro and var names containing underscores.
Expand All @@ -31,7 +31,7 @@ def markdown(s)
l = l.gsub(/ -- /, ' – ')
end
# Link function names to their definition within the page
l = l.gsub(/`(RedisModule_[A-z0-9]+)[()]*`/) {|x|
l = l.gsub(/`(ValkeyModule_[A-z0-9]+)[()]*`/) {|x|
$index[$1] ? "[#{x}](\##{$1})" : x
}
newlines << l
Expand Down Expand Up @@ -69,18 +69,18 @@ def linebreak_proto(proto, indent)
# Given the source code array and the index at which an exported symbol was
# detected, extracts and outputs the documentation.
def docufy(src,i)
m = /RM_[A-z0-9]+/.match(src[i])
name = m[0]
name = name.sub("RM_","RedisModule_")
m = /VM_[A-z0-9]+/.match(src[i])
shortname = m[0].sub("VM_","")
name = "ValkeyModule_" ++ shortname
proto = src[i].sub("{","").strip+";\n"
proto = proto.sub("RM_","RedisModule_")
proto = proto.sub("VM_","ValkeyModule_")
proto = linebreak_proto(proto, " ");
# Add a link target with the function name. (We don't trust the exact id of
# the generated one, which depends on the Markdown implementation.)
puts "<span id=\"#{name}\"></span>\n\n"
puts "### `#{name}`\n\n"
puts " #{proto}\n"
puts "**Available since:** #{$since[name] or "unreleased"}\n\n"
puts "**Available since:** #{$since[shortname] or "unreleased"}\n\n"
comment = ""
while true
i = i-1
Expand Down Expand Up @@ -132,7 +132,7 @@ def is_section_doc(src, i)

def is_func_line(src, i)
line = src[i]
return line =~ /RM_/ &&
return line =~ /VM_/ &&
line[0] != ' ' && line[0] != '#' && line[0] != '/' &&
src[i-1] =~ /\*\//
end
Expand All @@ -142,9 +142,7 @@ def is_func_line(src, i)
puts "linkTitle: \"API reference\"\n"
puts "weight: 1\n"
puts "description: >\n"
puts " Reference for the Redis Modules API\n"
puts "aliases:\n"
puts " - /topics/modules-api-ref\n"
puts " Reference for the Valkey Modules API\n"
puts "---\n"
puts "\n"
puts "<!-- This file is generated from module.c using\n"
Expand All @@ -155,22 +153,22 @@ def is_func_line(src, i)
$index = {}
src.each_with_index do |line,i|
if is_func_line(src, i)
line =~ /RM_([A-z0-9]+)/
name = "RedisModule_#{$1}"
line =~ /VM_([A-z0-9]+)/
name = "ValkeyModule_#{$1}"
$index[name] = true
end
end

# Populate the 'since' map (name => version) if we're in a git repo.
$since = {}
require File.dirname(__FILE__) ++ '/module-api-since.rb'
git_dir = File.dirname(__FILE__) ++ "/../.git"
if File.directory?(git_dir) && `which git` != ""
`git --git-dir="#{git_dir}" tag --sort=v:refname`.each_line do |version|
next if version !~ /^(\d+)\.\d+\.\d+?$/ || $1.to_i < 4
version.chomp!
`git --git-dir="#{git_dir}" cat-file blob "#{version}:src/module.c"`.each_line do |line|
if line =~ /^\w.*[ \*]RM_([A-z0-9]+)/
name = "RedisModule_#{$1}"
if line =~ /^\w.*[ \*]VM_([A-z0-9]+)/
name = $1
if ! $since[name]
$since[name] = version
end
Expand Down

0 comments on commit 6e05d0f

Please sign in to comment.