[FR] Implement full support for updating literature notes with grouped annotations in bullet lists

[FeralFlora]

Since update 1.0.2, Update Literature Note partially works for grouped list items using the templates I've shared before. But there are still issues:

Here are some scenarios:

1. No new annotations.
   • This now works as expected. Nothing is added (no duplicate annotations).
2. New annotation
   • The literature note is updated, but the new annotations are not placed under the existing heading where they belong. Duplicate headings are created.
3. A comment is added to an existing annotation:
   • The comment is not added.
4. An existing comment is changed.
   • The comment is not updated.

Note: I tested point 3. and 4. with the In-place Update of Existing Annotation toggled on.

Regarding point 2.:
I tried to make an zt-annots.eta template that could fix this issue, by keeping track of which headings have already been created, and only creating them if they don't exist. However, it doesn't work and I think it's because the createdColorLabels map doesn't persist.

Here is the template:

<%
// Define a map to keep track of which color labels have been created
const createdColorLabels = new Map();
// Loop over each annotation and group them by color
const colorGroups = it.group(annotation => annotation.colorName);

// Define the color label map
const colorLabel = {
  'blue': '⚡ Hypotheses',
  'green': '💡 Main ideas and conclusions',
  'orange': '⚙️ Method',
  'yellow': '⭐ Important',
  'purple': '🧩 Definitions and concepts',
  'red': '⛔ Weaknesses and caveats',
  'gray': '📣 Survey instruments'
};


// Loop over each color group and output the annotations
for (const [color, annotations] of Object.entries(colorGroups)) {
  const colorHeading = colorLabel[color] ?? 'Group';

// Check if the heading for this color has already been created
if (!createdColorLabels.has(colorHeading)) {%>
\n
### <%= colorHeading %>
\n
<% createdColorLabels.set(colorHeading, true); %>
<% }
  
// Output the annotations for this color
for (const annotation of annotations) { %>
<%~ include("annotation", annotation) %>
<% } %>
<% } %>

I had some ideas for persistence, either by storing the names of existing headings for each it.key in a separate file (JSON), or by checking level three headings in the existing literature note. However, I didn't manage to implement these ideas in the template successfully. I have several such experimental templates which are probably misguided.

[aidenlx]

I tried to make an zt-annots.eta template that could fix this issue, by keeping track of which headings have already been created, and only creating them if they don't exist. However, it doesn't work and I think it's because the createdColorLabels map doesn't persist.

The in-place update doesn't use zt-annots.eta, it only update a single annotation with zt-annot.eta

the in-place update is actually implement differently than simple overwrite updates. It need to first locate the range where it should update. it's kind of tricky to update existing group in a generic way, since we can't tell which heading/list item is the container for the grouped items without explictly specifying it. Before I come up with a new API to handle this, you can use overwrite mode to update note properly.

[lisasuosuo]

现在这个分组注释还是会重复生成标题，请问有解决办法吗。如果使用覆盖模式，那就完全无法编辑文件了

[mariomui]

@aidenlx have you considered the way that Zotero integration does it?

There is a special keyword called persist notes where the overwrite will ignore anything commented out by % tags. 'Persistent block'

Another feature is to add only annotations since the last update date. 'Update by recent'

https://github.com/mgmeyers/obsidian-zotero-integration/blob/main/docs/Templating.md

The intent is to safely organize thoughts inside the imported annotations incrementally and non destructively.

As far as features are concerned I am far more interested in the 'update by recent' than the 'persistent block'

PS I am happy to work on this. Should I post this on discussions?

[FeralFlora]

the way that Zotero integration does it?

This could make the overwrite update mode less destructive, but it wouldn't enable all the update modes that Obsidian-Zotero offers or aims to offer for custom templates - especially not the in-place updating of annotations.

This is the most promising/innovative update mode, in my view.

If you do the basic persist block (following the Zotero Integration's docs' example) for all annotations, the subsequent updates will not get added to the same group heading as the existing annotations. The existing ones persist as is, and the new ones get appended at the end, creating duplicate headings in the case of annotations grouped by color.

Unless you are using some custom template that has different behavior?

There is this experimental template that puts every annotation in separate persist blocks: https://forum.obsidian.md/t/zotero-integration-import-templates/36310/107?u=feralflora

I am not sure of the merits of this approach, and it seems like it would be quite messy. And again, it does not allow in-place updating of existing annotations, as far as I can tell.

[FeralFlora]

Is there any change regarding how well the plugin can cope with updating non-callout annotations and highlight? In the plugin settings it says:

Note: If you disable callout warpping in annotation template, you need to make sure the block-id is added properly in the template.

However, it's not documented how block-ids are properly added in order for the updating feature to work with callout wrapping disabled. Could you clarify that?

[FeralFlora]

@aidenlx?

[vxm5091]

@FeralFlora did you end up figuring it out? Dealing with the same issue

[lisasuosuo]

I have tried use dataview to update and sort the hightlight correctly. And now I am satisfied with the outcome .

.I use callout to hide the annotation and then use dataviewjs to show it .
This is my note.eta script ,hope it can help you.:

## 划线

>[!文献划线]
> ```dataviewjs
> const sortOrder = {
>   "重要/问题": 1,
>   "描述/概念": 2,
>   "思考": 3,
>   "案例": 4,
>   "语法": 5,
>   "other": 6  
> };
> let rows = dv.current().file.lists
>   .map(L => {
>     let substringValue = L.text.substring(13, 20);
>     let B;
> 
>     if (substringValue === "#ff6666") {
>         B = "重要/问题";
>     } else if (substringValue === "#ffd400") {
>         B = "描述/概念";
>     } else if (substringValue === "#5fb236") {
> 	    B = "思考";
> 	} else if (substringValue === "#2ea8e5") {
> 	    B = "案例";
>     } else if (substringValue === "#a28ae5") {
> 	    B = "语法";
>     } else {
>         B = "other";
>     }
> 
>     
>     let cleanedText = L.text.replace(/^>\s*\^[^\n]*/gm, "");
>     cleanedText = cleanedText.replace(/^>\s*💬/gm,"- 💬");
> 
>    
>     return [B, cleanedText];
>   }).sort((a, b) => {
>     return (sortOrder[a[0]] || 999) - (sortOrder[b[0]] || 999);
>   });
> 
>
> dv.table(["Type", "Highlight"], rows);
> ```

>[!quote]-
><%~ include("annots", it.annotations) %>