Commit Graph

1054 Commits

Author SHA1 Message Date
Anders Kaseorg 178736c8eb docs: Fix spelling errors caught by codespell.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-26 09:31:08 -07:00
Mateusz Mandera 1a8ad796f8 models: Replace __id syntax with _id where possible.
model__id syntax implies needing a JOIN on the model table to fetch the
id. That's usually redundant, because the first table in the query
simply has a 'model_id' column, so the id can be fetched directly.
Django is actually smart enough to not do those redundant joins, but we
should still avoid this misguided syntax.

The exceptions are ManytoMany fields and queries doing a backward
relationship lookup. If "streams" is a many-to-many relationship, then
streams_id is invalid - streams__id syntax is needed. If "y" is a
foreign fields from X to Y:
class X:
  y = models.ForeignKey(Y)

then object x of class X has the field x.y_id, but y of class Y doesn't
have y.x_id. Thus Y queries need to be done like
Y.objects.filter(x__id__in=some_list)
2021-04-22 14:53:00 -07:00
sahil839 4ac3fabadd models: Add new helper can_move_messages_between_streams.
This commit adds new helper can_move_messages_between_streams
which will be used to check whether a user is allowed to move
messages from one stream to another according to value of
'move_messages_between_streams_policy'.
2021-04-16 15:16:08 -07:00
sahil839 2dc99aa90f settings: Add new setting for controlling who can move msgs to stream.
This commit adds a new setting 'move_messages_between_streams_policy`
for controlling who can move messages from one stream to other.
2021-04-16 15:10:39 -07:00
Vishnu KS e72dea1de6 models: Create get_first_human_user function in Realm. 2021-04-16 13:22:01 -07:00
Anders Kaseorg e7ed907cf6 python: Convert deprecated Django ugettext alias to gettext.
django.utils.translation.ugettext is a deprecated alias of
django.utils.translation.gettext as of Django 3.0, and will be removed
in Django 4.0.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-04-15 18:01:34 -07:00
Aman Agrawal 802c450b3f realm: Add setting to configure GIPHY rating.
Organization admins can use this setting to restrict the maximum
rating of GIFs that will be retrieved from GIPHY. Also, there
is option to disable GIPHY too.
2021-04-14 10:29:39 -07:00
sahil839 d8c58527cc models: Remove unused function can_access_all_realm_members.
This commit removes can_access_all_realm_members function as
it is not used anywhere in code other than tests.

This function was originally added in 4483e33102 and was
only used in digest.py other than the tests, but its use
in diget.py was removed in 735b6cb761 and the function
itself was not removed from models.py.
2021-04-13 17:49:36 -07:00
Tim Abbott 2e928a0853 markdown: Remove logic for creating markdown engines for all realms.
This logic likely never ran due to a combination of bugs.

* Running `maybe_update_markdown_engines` unconditionally meant that
  `if md_engine_key in md_engines` was likely always true.
* Introduced in 65838bb: DEFAULT_MARKDOWN_KEY could never be in
  md_engines, so should we have ever reached that code path, we'd have
  tried to rebuild all markdown engines every time.

And it also wasn't clearly helpful -- because we fetch all linkifiers
for a realm on every request anyway, we don't really save database
queries by doing a bulk fetch on startup, and doing so would likely
result in a material regression to Zulip's overall startup time that
we were creating markdown engines for large numbers of realms in bulk
during process startup.
2021-04-13 09:18:18 -07:00
Abhijeet Prasad Bodas b140c17441 mute user: Cache list of muter IDs.
This commit defines a new function `get_muting_users`
which will return a list of IDs of users who have muted
a given user.
Whenever someone mutes/unmutes  a user, the cache will be
flushed, and subsequently when that user sends a message,
the cache will be populated with the list of people who
have muted them (maybe empty).

This data is a good candidate for caching because-

1. The function will later be called from the message send
codepath, and we try to minimize database queries there.

2. The entries will be pretty tiny.

3. The entries won't churn too much. An average user will
send messages much more frequently than get muted/unmuted,
and the first time penalty of hitting the db and populating
the cache should ideally get amortized by avoiding several
DB lookups on subsequent message sends.

The actual code to call this function will be written in
further commits.
2021-04-13 09:08:47 -07:00
Abhijeet Prasad Bodas 9602aa1467 mute user: Record entries in RealmAuditLog.
This makes it so that RealmAuditLog entries are
created when a user mutes/unmutes someone.

We don't really need to store the time, but we
do so anyways, because the `event_time` field
is currently a non-nullable one in the `RealmAuditLog`
model, and making it nullable would risk allowing
not specifying the time in other more important
code which also creates `RealmAuditLog` entries.

This also fixes an incorrect test of successfully
unmuting with the API. Earlier it did not mock
the time in the `views/muting.py` code to return
`mute_time`.
2021-04-13 09:08:47 -07:00
Tim Abbott cdbcb43706 api: Fix encoding of strings in realm endpoint.
* Don't require strings to be unnecessarily JSON-encoded.
* Use check_capped_string rather than custom code for length checks.
* Update frontend to pass the right parameters.

With a much simplified populate_data_for_request design suggested by
Anders; we only support a handful of data types, all of which are
correctly encoded automatically by jQuery.

Fixes part of #18035.
2021-04-09 16:27:30 -07:00
sahil839 554ce5bf0d realm: Add moderators and full members option in invite_to_realm_policy.
We add moderators and full members option to invite_to_realm_policy
by using COMMON_POLICY_TYPES and use can_invite_others_to_realm helper
added in previous commit. This commit only does the backend work,
frontend work will be done in separate commit.
2021-04-07 09:05:16 -07:00
sahil839 d6165225c4 models: Add can_invite_others_to_realm for checking who can send invites.
This commit adds can_invite_others_to_realm helper which will be used in
further in next commit when invite_to_realm_policy will be modified to
support all values of COMMON_POLICY_TYPES.

It is important for this commit's correctness that
INVITE_TO_REALM_POLICY_TYPES was initialized to use the same values.
2021-04-07 09:04:47 -07:00
sahil839 4c8339fa8c settings: Replace invite_by_admins_policy with invite_to_realm_policy.
This commit replaces invite_by_admins_policy, which was a bool field,
with a new enum field invite_by_realm_policy.

Though the final goal is to add moderators and full members option
using COMMON_POLICY_TYPES, but this will be done in a separate
commit to make this easy for review.
2021-04-07 09:02:33 -07:00
Sumanth V Rao 1ac8fe7538 events/tests/api: Send realm_playground events to clients.
We send the whole data set as a part of the event rather than
doing an add/remove operation for couple of reasons:
    * This would make the client logic simpler.
    * The playground data is small enough for us to not worry
      about performance.

Tweaked both `fetch_initial_state_data` and `apply_events` to
handle the new playground event.

Tests added to validate the event matches the expected schema.

Documented realm_playgrounds sections inside /events and
/register to support our openapi validation system in test_events.

Tweaked other tests like test_event_system.py and test_home.py
to account for the new event being generated.

Lastly, documented the changes to the API endpoints in
api/changelog.md and bumped API_FEATURE_LEVEL.

Tweaked by tabbott to add an `id` field in RealmPlayground objects
sent to clients, which is essential to sending the API request to
remove one.
2021-04-06 20:56:58 -07:00
Sumanth V Rao 40228972b9 models/realm: Add a model for storing realm playground information.
Tweaked exports.py to add the config object there so that our export
tool can include the table when exporting. Also includes all the
changes required to import the new table from the exported data.

Helper function `get_realm_playgrounds` added to fetch all
playgrounds in a realm.

Tests amended.
2021-04-07 08:20:53 +05:30
Abhijeet Prasad Bodas 3bfcaa3968 mute user: Add backend infrastructure code.
Adds backend code for the mute users feature.
This is just infrastructure work (database
interactions, helpers, tests, events, API docs
etc) and does not involve any behavioral/semantic
aspects of muted users.

Adds POST and DELETE endpoints, to keep the
URL scheme mostly consistent in terms of `users/me`.

TODOs:
1. Add tests for exporting `zulip_muteduser` database table.
2. Add dedicated methods to python-zulip-api to be used
   in place of the current `client.call_endpoint` implementation.
2021-04-06 18:44:08 -07:00
Abhijeet Prasad Bodas 89f6139505 mute user: Add model and makemigrations.
This commit adds a new database table to support
muting users, and generates a migration file for
the same.
2021-04-06 18:44:08 -07:00
Abhijeet Prasad Bodas 52a86d9604 linkifiers: Use dictionaries for internal structures.
This change does not affect the API in anyway.
All internal code now uses dictionaries to
denote a linkifier, instead of tuples.
2021-04-05 18:16:08 -07:00
Abhijeet Prasad Bodas 68fe912c63 refactor: Rename most of "filter" to "linkifier".
After this only the database table, events,
and API endpoints remain.
2021-04-05 18:14:07 -07:00
Abhijeet Prasad Bodas f896a7667f refactor: Update some uses of "filter" to "linkifier".
This updates some comments and local variables
which could be changed without breaking other
stuff.
2021-04-05 18:14:07 -07:00
Siddharth Asthana 47e478945b support: Create RealmAuditLog when updating realm_subdomain. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 80c9243c6a support: Create RealmAuditLog when updating sponsorship status. 2021-04-03 08:01:46 -07:00
Siddharth Asthana ddbc6d7662 support: Create RealmAuditLog when realm reactivation email is sent. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 233c4d520c support: Create RealmAuditLog when updating billing_method.
This commit also makes acting_user as a mandantory argument and fixes
the tests accordingly.
2021-04-03 08:01:46 -07:00
Siddharth Asthana 44c34cb39a support: Create RealmAuditLog when approving sponsorship. 2021-04-03 08:01:46 -07:00
Siddharth Asthana 6945ed3587 support: Pass acting_user to attach_discount_to_realm. 2021-04-03 08:01:46 -07:00
WookieMonkeys 1b6f68bb59 stream: Add entropy to deactivated streams.
Adding an additional `!` to the stream name each time a stream is
deactivated, to a maximum of 21 times, effectively limits number of
times a stream with a given name can be deactivated.  This is unlikely
to come up in common usage, but may be confusing when testing.

Change what we prepend to deactivated stream names to something with
more entropy than just `!`, by instead prepending a substring of hash
of the stream's ID.  `!`s.  Using 128 bits of the hash means that it
will require more than 10^18th renames to have a 1% chance of collision.

Because too-long stream names are also truncated at 60 characters,
having this entropy in the beginning of the name also helps address
potential issues from stream names that differed only in, e.g. the
60th character.

Fixes #17016.
2021-04-01 17:16:35 -07:00
Mateusz Mandera 1e37fde59f migrations: Subscription.is_user_active denormalization - step two.
This adds a migration to do the backfill and add the index that'll be
used to optimize the bulk_get_subscriber_user_ids query in the next
commit.
2021-03-30 09:29:36 -07:00
Mateusz Mandera f329878376 migrations: Subscription.is_user_active denormalization - step one.
This adds the is_user_active with the appropriate code for setting the
value correctly in the future. In the following commit a migration to
backfill the value for existing Subscriptions will be added.

To ensure correct user_profile.is_active handling also in tests, we
replace all direct .is_active mutation with calls to appropriate
functions.
2021-03-30 09:19:03 -07:00
sahil839 54be0dd1a4 streams: Add moderators option in stream_post_policy.
This commit adds a new option of STREAM_POST_POLICY_MODERATORS
in stream_post_policy which will allow only realm admins and
moderators to post in that stream.
2021-03-30 09:06:20 -07:00
sahil839 5b32dcd2e7 settings: Add moderators-only option in create_stream_policy.
This commit modifies the has_permission function to include
realm moderator role. Thus this adds a new option of moderators
only for create_stream_policy.
Though this automatically adds this option for invite_to_stream_policy
also, but we will keep other code for showing error and for tests
in a separate commit.
2021-03-30 09:06:20 -07:00
sahil839 911854d0bf models: Add assert statement for full_members policy in has_permission.
This commit adds an assert statement in the last block of
has_permission which checks whether the policy_value is
POLICY_FULL_MEMBERS_ONLY. This assert statement is added
for readability.
2021-03-30 09:06:20 -07:00
tushar912 2cf51139cf custom profile fields: Rename "CHOICE" to "SELECT" in frontend.
This requires a small backend change to the label.
2021-03-26 11:49:11 -07:00
Riken Shah b8691fb252 models: Add `clear_client_cache` function to clear `get_client_cache`.
This is a prep commit, this function will be used when we
want to reset the test environment of the puppeteer test.
2021-03-25 12:58:36 -07:00
Aman Agrawal 4d43a1baa9 default_stream_groups: Keep list of streams sorted by stream name.
Since the list of streams returned by a query which is not sorted
can vary, the tests which use it become flaky.
NormalActionsTest.test_default_stream_groups_events became
flaky due to this and hopefully sorting the streams should
fix it.
2021-03-25 14:44:26 +05:30
Mateusz Mandera 82d6d925e5 cache: Delete user_profile_by_email_cache_key.
This is no longer used in any important place,
get_user_profile_by_email is meant to be used only in manage.py shell
now and thus there's no point in this function being cached.
2021-03-25 00:47:42 -07:00
Tim Abbott d89c405074 custom_profile_fields: Revert incorrect early-converted hunk.
In b220d29fed, we incorrectly migrated a
field exposed to the frontend, which resulted in the frontend
puppeteer tests failing in CI.
2021-03-24 14:22:08 -07:00
tushar912 98a6bdbd4c custom profile fields: Rename "SELECT" field validator.
Rename the "SELECT" field validator so that it can be reused
with the upcoming "SELECT_MULTIPLE" field.
2021-03-24 12:54:51 -07:00
tushar912 b220d29fed custom profile fields: Rename "CHOICE" to "SELECT" in backend.
Rename the "CHOICE" field to "SELECT" in backend. This is
done to improve readability as a prep for the upcoming
"SELECT_MULTIPLE" field.
2021-03-24 12:54:51 -07:00
Vishnu KS 5e0030927b models: Make get_human_admin_users optionally not return realm owners. 2021-03-22 18:10:03 -07:00
Vishnu KS 7a351edb60 models: Make get_admin_users_and_bots optionally not return owners. 2021-03-22 18:10:03 -07:00
Aman Agrawal e587c029f6 display_settings: Allow user to set default_view.
TextField is used to allow users to set long stream + topic narrow
names in the urls.

We currently restrict users to only set "all_messages" and
"recent_topics" as narrows.

This commit achieves 3 things:
* Removes recent topics as the default view which loads when
  hash is empty.
* Loads default_view when hash is empty.
* Loads default_view on pressing escape key when it is unhandled by
  other present UI elements.

NOTE: After this commit loading zulip with an empty hash will
automatically set hash to default_view.  Ideally, we'd just display
the default view without a hash, but that involves extra complexity.

One exception is when user is trying to load an overlay directly,
i.e. zulip is loaded with an overlay hash. In this case,
we render recent topics is background irrespective of default_view.

We consider this last detail to be a bug not important enough to block
adding this setting.
2021-03-11 18:09:08 -08:00
sahil839 4ca21a6982 users: Give moderators same permissions as that of full members.
This commit updates the stream creation, subscribing others to
stream, wildcard mention settings and stream post policy to allow
realm moderators even if they are new and the respective setting
is set to allow full members only.
2021-03-02 17:19:31 -08:00
sahil839 b4fd15d516 models: Rename is_new_member to is_provisional_member.
This commit renames the is_new_member property in models.py
to is_provisional_member which will return true for any user
who is not a full member. We will add a condition in further
commit such that this returns 'False' for a moderator as we
will initially give all the rights to moderator that a full
member has.
2021-03-02 17:19:31 -08:00
Mateusz Mandera 6c7eee0a1f presence: Document that a user can disable their presence updates.
This is a crucial detail, as it makes UserPresence an unreliable
indicator of when users were recently active. It should be documented
more clearly.
2021-03-01 15:05:17 -08:00
sahil839 af64ca388a models: Add realm moderator role.
Have not included "ROLE_MODERATOR" in UserProfile.ROLE_TYPES
in this commit because did not want to update the openapi
docs at this stage as it will be a user-facing change and
not updating the openapi docs with moderator role included in
UserProfile.ROLE_TYPES gives error in ./tools/check-schemas.
2021-02-23 15:00:09 -08:00
Anders Kaseorg 6e4c3e41dc python: Normalize quotes with Black.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00
Anders Kaseorg 11741543da python: Reformat with Black, except quotes.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-12 13:11:19 -08:00