Archives
/
zk
mirror of https://github.com/mickael-menu/zk
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Support multiple `--match` flags (#268)

Browse Source
pull/274/head
Oliver Marriott 1 year ago committed by GitHub
parent 9d88245102
commit 142b636342
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
15 changed files with 139 additions and 64 deletions
Show Stats Download Patch File Download Diff File

1
CHANGELOG.md
Unescape Escape View File

@ -15,6 +15,7 @@ All notable changes to this project will be documented in this file.
Format a date returned by `get-date`:
{{date (get-date "monday") "timestamp"}}
```
* `zk list` now support multiple `--match`/`-m` flags, which allows to search for several tokens appearing in any order in the notes (contributed by [@rktjmp](https://github.com/mickael-menu/zk/pull/268)).
### Fixed

49
docs/editors-integration.md
Unescape Escape View File

@ -184,30 +184,31 @@ This LSP command calls `zk list` to search a notebook. It takes two arguments:
1. A path to any file or directory in the notebook, to locate it.
2. <details><summary>A dictionary of additional options (click to expand)</summary>
| Key | Type | Required? | Description |
|------------------|--------------|-----------|-------------------------------------------------------------------------|
| `select` | string array | Yes | List of note fields to return<sup>1</sup> |
| `hrefs` | string array | No | Find notes matching the given path, including its descendants |
| `limit` | integer | No | Limit the number of notes found |
| `match` | string | No | Terms to search for in the notes |
| `exactMatch` | boolean | No | Search for exact occurrences of the `match` argument (case insensitive) |
| `excludeHrefs` | string array | No | Ignore notes matching the given path, including its descendants |
| `tags` | string array | No | Find notes tagged with the given tags |
| `mention` | string array | No | Find notes mentioning the title of the given ones |
| `mentionedBy` | string array | No | Find notes whose title is mentioned in the given ones |
| `linkTo` | string array | No | Find notes which are linking to the given ones |
| `linkedBy` | string array | No | Find notes which are linked by the given ones |
| `orphan` | boolean | No | Find notes which are not linked by any other note |
| `related` | string array | No | Find notes which might be related to the given ones |
| `maxDistance` | integer | No | Maximum distance between two linked notes |
| `recursive` | boolean | No | Follow links recursively |
| `created` | string | No | Find notes created on the given date |
| `createdBefore` | string | No | Find notes created before the given date |
| `createdAfter` | string | No | Find notes created after the given date |
| `modified` | string | No | Find notes modified on the given date |
| `modifiedBefore` | string | No | Find notes modified before the given date |
| `modifiedAfter` | string | No | Find notes modified after the given date |
| `sort` | string array | No | Order the notes by the given criterion |
| Key | Type | Required? | Description |
| ------------------ | -------------- | ----------- | ------------------------------------------------------------------------- |
| `select` | string array | Yes | List of note fields to return<sup>1</sup> |
| `hrefs` | string array | No | Find notes matching the given path, including its descendants |
| `limit` | integer | No | Limit the number of notes found |
| `match` | string array | No | Terms to search for in the notes |
| `exactMatch` | boolean | No | (deprecated: use `matchStrategy`) Search for exact occurrences of the `match` argument (case insensitive) |
| `matchStrategy` | string | No | Specify match strategy, which may be "fts" (default), "exact" or "re" |
| `excludeHrefs` | string array | No | Ignore notes matching the given path, including its descendants |
| `tags` | string array | No | Find notes tagged with the given tags |
| `mention` | string array | No | Find notes mentioning the title of the given ones |
| `mentionedBy` | string array | No | Find notes whose title is mentioned in the given ones |
| `linkTo` | string array | No | Find notes which are linking to the given ones |
| `linkedBy` | string array | No | Find notes which are linked by the given ones |
| `orphan` | boolean | No | Find notes which are not linked by any other note |
| `related` | string array | No | Find notes which might be related to the given ones |
| `maxDistance` | integer | No | Maximum distance between two linked notes |
| `recursive` | boolean | No | Follow links recursively |
| `created` | string | No | Find notes created on the given date |
| `createdBefore` | string | No | Find notes created before the given date |
| `createdAfter` | string | No | Find notes created after the given date |
| `modified` | string | No | Find notes modified on the given date |
| `modifiedBefore` | string | No | Find notes modified before the given date |
| `modifiedAfter` | string | No | Find notes modified after the given date |
| `sort` | string array | No | Order the notes by the given criterion |
1. As the output of this command might be very verbose and put a heavy load on the LSP client, you need to explicitly set which note fields you want to receive with the `select` option. The following fields are available: `filename`, `filenameStem`, `path`, `absPath`, `title`, `lead`, `body`, `snippets`, `rawContent`, `wordCount`, `tags`, `metadata`, `created`, `modified` and `checksum`.

15
docs/note-filtering.md
Unescape Escape View File

@ -58,6 +58,21 @@ Change the currently used strategy with `--match-strategy <strategy>` (or `-M`).
list = "zk list --match-strategy re $@"
```
The `--match` option may be given multiple times, where each argument will be combined with a boolean AND.
For example,
```sh
$ zk list --tag "recipe" --match "pizza -pineapple" --match "mushrooms"
```
Is equivalent to,
```sh
$ zk list --tag "recipe" --match "(pizza -pineapple) AND (mushrooms)"
```
### Full-text search (`fts`)
The default match strategy is powered by a [full-text search](https://en.wikipedia.org/wiki/Full-text_search) database enabling near-instant results. Queries are not case-sensitive and terms are tokenized, which means that searching for `create` will also match `created` and `creating`.

5
internal/adapter/handlebars/handlebars_test.go
Unescape Escape View File

@ -238,12 +238,13 @@ func TestDateHelper(t *testing.T) {
testString(t, "{{date now 'timestamp'}}", context, "200911172034")
testString(t, "{{date now 'timestamp-unix'}}", context, "1258490098")
testString(t, "{{date now 'cust: %Y-%m'}}", context, "cust: 2009-11")
testString(t, "{{date now 'elapsed'}}", context, "13 years ago")
testString(t, "{{date now 'elapsed'}}", context, "14 years ago")
}
func TestGetDateHelper(t *testing.T) {
context := map[string]interface{}{"now": time.Date(2009, 11, 17, 20, 34, 58, 651387237, time.UTC)}
testString(t, "{{get-date \"2009-11-17T20:34:58\"}}", context, "2009-11-17 20:34:58 +0000 UTC")
localOffsetAndTZ := time.Now().Format("-0700 MST")
testString(t, "{{get-date \"2009-11-17T20:34:58\"}}", context, "2009-11-17 20:34:58 "+localOffsetAndTZ)
}
func TestShellHelper(t *testing.T) {

25
internal/adapter/sqlite/note_dao.go
Unescape Escape View File

@ -12,7 +12,6 @@ import (
"github.com/mickael-menu/zk/internal/util"
"github.com/mickael-menu/zk/internal/util/errors"
"github.com/mickael-menu/zk/internal/util/fts5"
"github.com/mickael-menu/zk/internal/util/opt"
"github.com/mickael-menu/zk/internal/util/paths"
strutil "github.com/mickael-menu/zk/internal/util/strings"
)
@ -421,9 +420,7 @@ func (d *NoteDAO) expandMentionsIntoMatch(opts core.NoteFindOpts) (core.NoteFind
}
// Expand the mention queries in the match predicate.
match := opts.Match.String()
match += " " + strings.Join(mentionQueries, " OR ")
opts.Match = opt.NewString(match)
opts.Match = append(opts.Match, " ("+strings.Join(mentionQueries, " OR ")+") ")
return opts, nil
}
@ -519,20 +516,26 @@ func (d *NoteDAO) findRows(opts core.NoteFindOpts, selection noteSelection) (*sq
return nil
}
if !opts.Match.IsNull() {
if 0 < len(opts.Match) {
switch opts.MatchStrategy {
case core.MatchStrategyExact:
whereExprs = append(whereExprs, `n.raw_content LIKE '%' || ? || '%' ESCAPE '\'`)
args = append(args, escapeLikeTerm(opts.Match.String(), '\\'))
for _, match := range opts.Match {
whereExprs = append(whereExprs, `n.raw_content LIKE '%' || ? || '%' ESCAPE '\'`)
args = append(args, escapeLikeTerm(match, '\\'))
}
case core.MatchStrategyFts:
snippetCol = `snippet(fts_match.notes_fts, 2, '<zk:match>', '</zk:match>', '…', 20)`
joinClauses = append(joinClauses, "JOIN notes_fts fts_match ON n.id = fts_match.rowid")
additionalOrderTerms = append(additionalOrderTerms, `bm25(fts_match.notes_fts, 1000.0, 500.0, 1.0)`)
whereExprs = append(whereExprs, "fts_match.notes_fts MATCH ?")
args = append(args, fts5.ConvertQuery(opts.Match.String()))
for _, match := range opts.Match {
whereExprs = append(whereExprs, "fts_match.notes_fts MATCH ?")
args = append(args, fts5.ConvertQuery(match))
}
case core.MatchStrategyRe:
whereExprs = append(whereExprs, "n.raw_content REGEXP ?")
args = append(args, opts.Match.String())
for _, match := range opts.Match {
whereExprs = append(whereExprs, "n.raw_content REGEXP ?")
args = append(args, match)
}
break
}
}

23
internal/adapter/sqlite/note_dao_test.go
Unescape Escape View File

@ -312,7 +312,7 @@ func TestNoteDAOFindMinimalAll(t *testing.T) {
func TestNoteDAOFindMinimalWithFilter(t *testing.T) {
testNoteDAO(t, func(tx Transaction, dao *NoteDAO) {
notes, err := dao.FindMinimal(core.NoteFindOpts{
Match: opt.NewString("daily | index"),
Match: []string{"daily | index"},
MatchStrategy: core.MatchStrategyFts,
Sorters: []core.NoteSorter{{Field: core.NoteSortWordCount, Ascending: true}},
Limit: 3,
@ -368,7 +368,7 @@ func TestNoteDAOFindTag(t *testing.T) {
func TestNoteDAOFindMatch(t *testing.T) {
testNoteDAOFind(t,
core.NoteFindOpts{
Match: opt.NewString("daily | index"),
Match: []string{"daily | index"},
MatchStrategy: core.MatchStrategyFts,
},
[]core.ContextualNote{
@ -452,10 +452,25 @@ func TestNoteDAOFindMatch(t *testing.T) {
)
}
func TestNoteDAOFindMatchWithMultiMatch(t *testing.T) {
testNoteDAOFindPaths(t,
core.NoteFindOpts{
Match: []string{"daily | index", "second"},
MatchStrategy: core.MatchStrategyFts,
Sorters: []core.NoteSorter{
{Field: core.NoteSortPath, Ascending: false},
},
},
[]string{
"log/2021-01-04.md",
},
)
}
func TestNoteDAOFindMatchWithSort(t *testing.T) {
testNoteDAOFindPaths(t,
core.NoteFindOpts{
Match: opt.NewString("daily | index"),
Match: []string{"daily | index"},
MatchStrategy: core.MatchStrategyFts,
Sorters: []core.NoteSorter{
{Field: core.NoteSortPath, Ascending: false},
@ -474,7 +489,7 @@ func TestNoteDAOFindExactMatch(t *testing.T) {
test := func(match string, expected []string) {
testNoteDAOFindPaths(t,
core.NoteFindOpts{
Match: opt.NewString(match),
Match: []string{match},
MatchStrategy: core.MatchStrategyExact,
},
expected,

12
internal/cli/filtering.go
Unescape Escape View File

@ -9,7 +9,6 @@ import (
"github.com/mickael-menu/zk/internal/core"
dateutil "github.com/mickael-menu/zk/internal/util/date"
"github.com/mickael-menu/zk/internal/util/errors"
"github.com/mickael-menu/zk/internal/util/opt"
"github.com/mickael-menu/zk/internal/util/strings"
)
@ -19,7 +18,7 @@ type Filtering struct {
Interactive bool `kong:"group='filter',short='i',help='Select notes interactively with fzf.'" json:"-"`
Limit int `kong:"group='filter',short='n',placeholder='COUNT',help='Limit the number of notes found.'" json:"limit"`
Match string `kong:"group='filter',short='m',placeholder='QUERY',help='Terms to search for in the notes.'" json:"match"`
Match []string `kong:"group='filter',short='m',placeholder='QUERY',help='Terms to search for in the notes.'" json:"match"`
MatchStrategy string `kong:"group='filter',short='M',default='fts',placeholder='STRATEGY',help='Text matching strategy among: fts, re, exact.'" json:"matchStrategy"`
Exclude []string `kong:"group='filter',short='x',placeholder='PATH',help='Ignore notes matching the given path, including its descendants.'" json:"excludeHrefs"`
Tag []string `kong:"group='filter',short='t',help='Find notes tagged with the given tags.'" json:"tags"`
@ -117,11 +116,7 @@ func (f Filtering) ExpandNamedFilters(filters map[string]string, expandedFilters
f.ModifiedAfter = parsedFilter.ModifiedAfter
}
if f.Match == "" {
f.Match = parsedFilter.Match
} else if parsedFilter.Match != "" {
f.Match = fmt.Sprintf("(%s) AND (%s)", f.Match, parsedFilter.Match)
}
f.Match = append(f.Match, parsedFilter.Match...)
if f.MatchStrategy == "" {
f.MatchStrategy = parsedFilter.MatchStrategy
}
@ -148,7 +143,8 @@ func (f Filtering) NewNoteFindOpts(notebook *core.Notebook) (core.NoteFindOpts,
return opts, fmt.Errorf("the --exact-match (-e) option is deprecated, use --match-strategy=exact (-Me) instead")
}
opts.Match = opt.NewNotEmptyString(f.Match)
opts.Match = make([]string, len(f.Match))
copy(opts.Match, f.Match)
opts.MatchStrategy, err = core.MatchStrategyFromString(f.MatchStrategy)
if err != nil {
return opts, err

6
internal/cli/filtering_test.go
Unescape Escape View File

@ -11,7 +11,7 @@ func TestExpandNamedFiltersNone(t *testing.T) {
Path: []string{"path1"},
Limit: 10,
Interactive: true,
Match: "match query",
Match: []string{"match query"},
Exclude: []string{"excl-path1", "excl-path2"},
Tag: []string{"tag1", "tag2"},
Mention: []string{"mention1", "mention2"},
@ -156,7 +156,7 @@ func TestExpandNamedFiltersJoinLitterals(t *testing.T) {
func TestExpandNamedFiltersJoinMatch(t *testing.T) {
f := Filtering{
Path: []string{"f1", "f2"},
Match: "(chocolate OR caramel)",
Match: []string{"(chocolate OR caramel)"},
}
res, err := f.ExpandNamedFilters(
@ -168,7 +168,7 @@ func TestExpandNamedFiltersJoinMatch(t *testing.T) {
)
assert.Nil(t, err)
assert.Equal(t, res.Match, "(((chocolate OR caramel)) AND (banana)) AND (apple)")
assert.Equal(t, res.Match, []string{"(chocolate OR caramel)", "banana", "apple"})
}
func TestExpandNamedFiltersExpandsRecursively(t *testing.T) {

4
internal/core/note_find.go
Unescape Escape View File

@ -5,14 +5,12 @@ import (
"strings"
"time"
"unicode/utf8"
"github.com/mickael-menu/zk/internal/util/opt"
)
// NoteFindOpts holds a set of filtering options used to find notes.
type NoteFindOpts struct {
// Filter used to match the notes with the given MatchStrategy.
Match opt.String
Match []string
// Text matching strategy used with Match.
MatchStrategy MatchStrategy
// Filter by note hrefs.

7
internal/core/notebook.go
Unescape Escape View File

@ -235,13 +235,6 @@ func (n *Notebook) FindByHref(href string, allowPartialHref bool) (*MinimalNote,
})
}
// FindMatching retrieves the first note matching the given search terms.
func (n *Notebook) FindMatching(terms string) (*MinimalNote, error) {
return n.FindMinimalNote(NoteFindOpts{
Match: opt.NewNotEmptyString(terms),
})
}
// FindLinksBetweenNotes retrieves the links between the given notes.
func (n *Notebook) FindLinksBetweenNotes(ids []NoteID) ([]ResolvedLink, error) {
return n.index.FindLinksBetweenNotes(ids)

2
tests/cmd-graph.tesh
Unescape Escape View File

@ -24,7 +24,7 @@ $ zk graph --help
>Filtering
> -i, --interactive Select notes interactively with fzf.
> -n, --limit=COUNT Limit the number of notes found.
> -m, --match=QUERY Terms to search for in the notes.
> -m, --match=QUERY,... Terms to search for in the notes.
> -M, --match-strategy=STRATEGY Text matching strategy among: fts, re, exact.
> -x, --exclude=PATH,... Ignore notes matching the given path,
> including its descendants.

31
tests/cmd-list-filter-match-exact.tesh
Unescape Escape View File

@ -15,3 +15,34 @@ $ zk list -q --debug-style -Me --match '["न", "म", "स्", "ते"]'
>
> - Given the Hindi word "नमस्ते":
>
# Mutliple match flags.
$ zk list -q --debug-style -Me --match "thread" --match "mut"
><title>Concurrency in Rust</title> <path>g7qa.md</path> (just now)
>
> - * Thanks to the [Ownership pattern](88el), Rust has a model of [Fearless concurrency](2cl7).
> * Rust aims to have a small runtime, so it doesn't support [green threads](inbox/my59).
> * Crates exist to add support for green threads if needed.
> * Instead, Rust relies on the OS threads, a model called 1-1.
>
><title>Mutex</title> <path>inbox/er4k.md</path> (just now)
>
> - * Abbreviation of *mutual exclusion*.
> * An approach to manage safely shared state by allowing only a single thread to access a protected value at one time.
> * A mutex *guards* a protected data with a *locking system*.
> * Managing mutexes is tricky, using [channels](../fwsj) is an easier alternative.
> * The main risk is to create *deadlocks*.
> * Thanks to its [Ownership](../88el) pattern, Rust makes sure we can't mess up when using locks.
>
# Mutliple match flags.
$ zk list -q --debug-style -Me --match "thread" --match "mutual"
><title>Mutex</title> <path>inbox/er4k.md</path> (just now)
>
> - * Abbreviation of *mutual exclusion*.
> * An approach to manage safely shared state by allowing only a single thread to access a protected value at one time.
> * A mutex *guards* a protected data with a *locking system*.
> * Managing mutexes is tricky, using [channels](../fwsj) is an easier alternative.
> * The main risk is to create *deadlocks*.
> * Thanks to its [Ownership](../88el) pattern, Rust makes sure we can't mess up when using locks.
>

9
tests/cmd-list-filter-match-fts.tesh
Unescape Escape View File

@ -24,6 +24,15 @@ $ zk list -q --debug-style --match 'green channel'
> * Instead, Rust…
>
# Search for two term by two --match flags (implicit AND).
$ zk list -q --debug-style --match 'green' --match 'channel'
><title>Concurrency in Rust</title> <path>g7qa.md</path> (just now)
>
> - …runtime, so it doesn't support [<term>green</term> threads](inbox/my59).
> * Crates exist to add support for <term>green</term> threads if needed.
> * Instead, Rust…
>
# Search for two terms with explicit AND.
$ zk list -q --debug-style --match 'green AND channel'
><title>Concurrency in Rust</title> <path>g7qa.md</path> (just now)

12
tests/cmd-list-filter-match-re.tesh
Unescape Escape View File

@ -15,3 +15,15 @@ $ zk list -q --debug-style -Mr --match 'न.*ते'
>
> - Given the Hindi word "नमस्ते":
>
# multiple match flags.
$ zk list -q --debug-style -Mr --match "mut.*" --match "thr..d"
><title>Mutex</title> <path>inbox/er4k.md</path> (just now)
>
> - * Abbreviation of *mutual exclusion*.
> * An approach to manage safely shared state by allowing only a single thread to access a protected value at one time.
> * A mutex *guards* a protected data with a *locking system*.
> * Managing mutexes is tricky, using [channels](../fwsj) is an easier alternative.
> * The main risk is to create *deadlocks*.
> * Thanks to its [Ownership](../88el) pattern, Rust makes sure we can't mess up when using locks.
>

2
tests/cmd-list.tesh
Unescape Escape View File

@ -32,7 +32,7 @@ $ zk list --help
>Filtering
> -i, --interactive Select notes interactively with fzf.
> -n, --limit=COUNT Limit the number of notes found.
> -m, --match=QUERY Terms to search for in the notes.
> -m, --match=QUERY,... Terms to search for in the notes.
> -M, --match-strategy=STRATEGY Text matching strategy among: fts, re, exact.
> -x, --exclude=PATH,... Ignore notes matching the given path,
> including its descendants.

Write Preview
Loading…
Cancel
Save