Lookup subject | Permify | Page 1

zenith stirrup Oct 18, 2024, 5:39 PM

#

Hi @plain wadi, It's definitely not expected behavior. Could you send your schema and the request for us to be reviewed ?

plain wadi Oct 18, 2024, 5:40 PM

#

sure

#

schema:

#

    attribute first_name string
}

entity org {
    relation admin @user
    relation perms @group_perms
}

entity project {
    relation parent @org
    relation admin @user 
    relation perms @group_perms

    permission project_edit = admin or parent.admin or perms.project_edit
    permission project_view = project_edit or perms.project_view
}

entity group {
    relation member @user
}

entity group_perms {
    relation members @group

    attribute can_project_edit boolean
    attribute can_project_view boolean

    permission project_edit = can_project_edit and members.member
    permission project_view = (can_project_view and members.member) or project_edit
}```

#

I add 5000 tuples:

#

                        Subject: client.Subject{
                            Type: "user",
                            ID:   uid,
                        },
                        Relation: "admin",
                        Entity: client.Entity{
                            Type: "org",
                            ID:   orgID,
                        },
                    })```

#

and then LookupSubject...

#

                        TenantId: tenantID,
                        Metadata: &v1.PermissionLookupSubjectRequestMetadata{
                            //SnapToken: snapShot,
                            //SchemaVersion: sr.SchemaVersion, // sr --> schema write response
                            Depth: 5,
                        },
                        SubjectReference: &v1.RelationReference{
                            Type: "user",
                        },
                        Permission: "project_view",
                        Entity: &v1.Entity{
                            Type: "project",
                            Id:   projectID,
                        },
                        Context:         nil,
                        PageSize:        500,
                        ContinuousToken: ct,
                    })```

#

I also added some other tuples to set up the group_perms etc.

zenith stirrup Oct 18, 2024, 5:44 PM

#

Thanks for the details also which version are you currently using ?

plain wadi Oct 18, 2024, 5:46 PM

#

permify   | │                    Permify v1.1.3                      │
permify   | │          Fine-grained Authorization Service            │
permify   | │                                                        │
permify   | │    docs: ............... https://docs.permify.co       │
permify   | │    github: .. https://github.com/Permify/permify       │
permify   | │    blog: ............... https://permify.co/blog       │
permify   | │                                                        │
permify   | └────────────────────────────────────────────────────────┘```

#

with 155k tuples in postgres

zenith stirrup Oct 18, 2024, 5:47 PM

#

Could you try it again with the newest version ? 1.1.5.

plain wadi Oct 18, 2024, 5:47 PM

#

I could probably construct a gist - that reproduces it ?

#

ah ok

#

ok that is definitely behaving differently now!

#

There is now something funky with the ContinuousToken

zenith stirrup Oct 18, 2024, 5:57 PM

#

Whats the issue with the ContinuousToken?

plain wadi Oct 18, 2024, 5:57 PM

#

I'm getting the same token from each request

zenith stirrup Oct 18, 2024, 6:00 PM

#

Seems like its a bug, we're on it right now

#

I guess the first issue with LookupSubject is resolved right ?

#

Another question: Does this happens only with the attributes?

plain wadi Oct 18, 2024, 6:02 PM

#

Right .. I just dug into the call with the debugger and the ContinuousToken is being passed into the gRPC call - but the same token is being returned.

#

it might be of note that I am not specifying a SnapToken or SchemaVersion in the metadata

#

When I remove the attribute based permissions - i get the same behaviour with the ContinuousToken

#

OK - same problem with SnapToken and SchemaVersion set on the metadata

zenith stirrup Oct 18, 2024, 6:58 PM

#

When you use PageSize=500, are you getting exactly 500 results per page ?

plain wadi Oct 18, 2024, 9:44 PM

#

prior to 1.1.5 (without attribute based permissions) then yes the pages and final page count would mach the actual numbers. So yes I would get 500 per page until the last page

zenith stirrup Oct 19, 2024, 11:46 AM

#

Hi @plain wadi, we've tried the endpoints (mostly using curl, not the SDK) and didn't encounter any issues. If possible, could you also try without the SDK so that we can ensure the issue is related to the Go client?

plain wadi Oct 20, 2024, 12:13 PM

#

@zenith stirrup can confirm a custom Go http client works fine. So yah - looks like its in the SDK or gRPC stack.

plain wadi Oct 20, 2024, 12:41 PM

#

Ok - I spoke too soon! Something is wonky - the ContinuousToken is different on each iteration, BUT - paging is returning the wrong count. If I increase the page size to be larger than the expected result set, it returns the correct list.

plain wadi Oct 20, 2024, 1:43 PM

#

So! When I cause the page size (100) to be just smaller than the expected set of ids (103) - I get 134 ids back. When I set the page size to be 200 - i get the expected 103 ids back.

#

BUT - if i remove the attribute based permissions from the schema (keeping the tuples the same) then the paged list becomes correct! 2 calls returning 103 as expected.

#

I can also tell you that with the attribute based permissions on, the extra 31 ids none of them are duplicates of the expected 103,

#

The extra ids belong to users who should not have the searched for permission. When I do a check for these extra IDs (for the same permission I searched for) - the all fail the check as expected.

#

FWIW - This is reproducible from an empty DB.

deep viper Oct 20, 2024, 4:02 PM

#

Hi @plain wadi , we are reviewing it and will get back to you quickly

deep viper Oct 20, 2024, 5:06 PM

#

Can you explain why the extra IDs are incorrect results? Why shouldn’t these subjects be included?

plain wadi Oct 20, 2024, 5:09 PM

#

because they are the ids of users that are only admin of a different 'org' . - but more importantly - with no paging - they are not present, when paging they become present.

#

That is as long as i change the page size to be large enough to return the expected results in one page - everything is as expected, if i reduce the page size less than the expected set of IDs - then these spurious ids appear.

#

I could give you a DB dump, or perhaps try and set up a test in a fork - if that would help?

deep viper Oct 20, 2024, 5:22 PM

#

it would be great if you could provide a DB dump, as we can quickly run tests with it

plain wadi Oct 20, 2024, 5:43 PM

#

📎 postgres_localhost-2024_10_20_18_41_54-dump.sql

#

curl with all results on one page...

#

--header 'Content-Type: application/json' \
--data-raw '{
  "metadata":{
    "depth": 10
  },
  "entity": {
    "type": "project",
    "id": "30f64e70-6cbc-4afa-9109-8aeb6a076d94"
  },
  "permission": "project_view",
  "subject_reference":{
    "type": "user"
   },
   "page_size": 30,
   "continuous_token": ""
}'```

#

Same lookup, but page size 19 ... needs three pages ...

#

curl -v --request POST 'http://localhost:3476/v1/tenants/t1/permissions/lookup-subject' \
--header 'Content-Type: application/json' \
--data-raw '{
  "metadata":{
    "depth": 10
  },
  "entity": {
    "type": "project",
    "id": "30f64e70-6cbc-4afa-9109-8aeb6a076d94"
  },
  "permission": "project_view",
  "subject_reference":{
    "type": "user"
   },
   "page_size": 19,
   "continuous_token": ""
}'

curl -v --request POST 'http://localhost:3476/v1/tenants/t1/permissions/lookup-subject' \
--header 'Content-Type: application/json' \
--data-raw '{
  "metadata":{
    "depth": 10
  },
  "entity": {
    "type": "project",
    "id": "30f64e70-6cbc-4afa-9109-8aeb6a076d94"
  },
  "permission": "project_view",
  "subject_reference":{
    "type": "user"
   },
   "page_size": 19,
   "continuous_token": "MzdhYTI1ZjItZjY4My00MmQ5LTllZTAtNWEzYzJkM2EzYTk3"
}'


curl -v --request POST 'http://localhost:3476/v1/tenants/t1/permissions/lookup-subject' \
--header 'Content-Type: application/json' \
--data-raw '{
  "metadata":{
    "depth": 10
  },
  "entity": {
    "type": "project",
    "id": "30f64e70-6cbc-4afa-9109-8aeb6a076d94"
  },
  "permission": "project_view",
  "subject_reference":{
    "type": "user"
   },
   "page_size": 19,
   "continuous_token": "ZDQ2Njk1ODItY2M1MS00Y2QzLTk1YWQtM2I0NzFkZTlkZmU1"
}'

deep viper Oct 20, 2024, 6:08 PM

#

Thank you for sharing. I will test it with this and get back to you

deep viper Oct 20, 2024, 9:54 PM

#

Hi @plain wadi , we’ve found the source of the issue. We’ll add some new test cases. for subject filtering and release a new version tomorrow

plain wadi Oct 21, 2024, 7:09 AM

#

Awesome news

plain wadi Oct 21, 2024, 7:33 AM

#

For this kind of 'bulk/list' behaviour - would the advice be to try and segment our dataset by tenant - would that be better than having all tuples in a single tenant?

deep viper Oct 21, 2024, 8:23 AM

#

Can you test with this image? ghcr.io/permify/permify-beta:v1.1.6-fd010409

plain wadi Oct 21, 2024, 8:41 AM

#

will do

#

Wow! That was fast - YES it is now behaving as expected.

deep viper Oct 21, 2024, 8:53 AM

#

Our multi-tenant solution provides full isolation for access checks, but the answer to your question depends on the use case. Would you be open to a call at your convenience to discuss this further and assist with any other topics if needed?

plain wadi Oct 21, 2024, 12:44 PM

#

Yah a call would be good. For clarity - we are in the early stages of investigation.

deep viper Oct 21, 2024, 12:56 PM

#

great, you can schedule a call from: https://calendly.com/d/5hk-8db-vh2/call-with-an-expert

Calendly

Call with an Expert

Schedule a call with our experts to discover anything related to Authorization and Security.

plain wadi Oct 21, 2024, 7:10 PM

#

OK, I will discuss with our senior engineer for our iam team, and work out a time.

#Lookup subject