Commit Graph

68 Commits

Author SHA1 Message Date
Steve Howell 57efe9d81a performance: Streamline list_to_streams.
We take advantage of stream.recipient to simplify
the query's where clause and avoid the need
for select_related("recipient").
2020-10-16 12:58:11 -07:00
Steve Howell e1bcf6124f refactor: Remove recipient from access_stream_by_name. 2020-10-16 12:58:11 -07:00
Steve Howell a51b483f1a performance: Remove recipient from access_stream_by_id.
The Recipient table is now kind of useless for
stream-related operations, since we have
recipient_id on Stream now.
2020-10-16 12:58:11 -07:00
Steve Howell 31622feb87 refactor: Only return sub from access_stream_common.
Let the callers access stream.recipient as needed.
It costs the same, and some of the callers can
actually stop caring about the actual Recipient
object.
2020-10-16 12:58:11 -07:00
Steve Howell 3b338ec32e performance: Optimize filter_stream_authorization.
We no longer do O(N) queries to get existing streams.

This is a somewhat contrived use case--generally, we
are not trying to re-subscribe a user to several
streams.  Still, we want to avoid this.

This commit also makes `test_bulk_subscribe_many`
do more work, and the change to the test helped
me discover this bug.
2020-10-13 18:54:55 -04:00
sahil839 4e8f5b5b31 streams: Change access_stream_for_delete_or_update to also return sub.
We modify access_stream_for_delete_or_update function to return
Subscription object also along with stream. This change will be
helpful in avoiding an extra query to get subscription object in
code for updating subscription role.
2020-10-08 17:07:30 -07:00
Aman Agrawal 8b419c93e4
message_send: Fix old guests being treated as full members.
For streams in which only full members are allowed to post,
we block guest users from posting there.

Guests users were blocked from posting to admin only streams
already. So now, guest users can only post to
STREAM_POST_POLICY_EVERYONE streams.

This is not a new feature but a bugfix which should have
happened when implementing full member stream policy / guest users.
2020-10-08 11:30:11 -07:00
Steve Howell c199571112 mypy: Add StreamDict.
This requires us to rework the view code a little
bit to explicitly assign fields.
2020-09-29 16:49:10 -07:00
Clara Dantas 8674287192 digest: Support digest of web public streams for guest users.
This change requires some basic plumbing for test code creating
web-public streams.
2020-09-25 16:11:04 -07:00
Aman c114447e11 topic_history: Allow anonymous access in web-public streams.
For web-public streams, clients can access full topic history
without being authenticated. They only need to additionally
send "streams:web-public" narrow with their request like all
the other web-public queries.
2020-09-04 10:17:56 -07:00
sahil839 ca1a8ac78f streams: Allow stream admin to update and deactivate streams.
The new Stream administrator role is allowed to manage a stream they
administer, including:
* Setting properties like name, description, privacy and post-policy.
* Removing subscribers
* Deactivating the stream

The access_stream_for_delete_or_update is modified and is used only
to get objects from database and further checks for administrative
rights is done by check_stream_access_for_delete_or_update.

We have also added a new exception class StreamAdministratorRequired.
2020-08-12 17:02:01 -07:00
Aman Agrawal 9f9daeea5b message_fetch: Allow access to web-public msgs for unauth users.
Via API, users can now access messages which are in web-public
streams without any authentication.

If the user is not authenticated, we assume it is a web-public
query and add `streams:web-public` narrow if not already present
to the narrow. web-public streams are also directly accessible.

Any malformed narrow which is not allowed in a web-public query
results in a 400 or 401. See test_message_fetch for the allowed
queries.
2020-08-12 16:39:02 -07:00
Alex Vandiver 39368cad3a tornado: Extract functions called from django into one module.
This makes clearer the separation of concerns.
2020-08-10 16:55:56 -07:00
Clara Dantas 0994b029d6 streams: Grant authorization to guest users to subscribe.
Modifies filter_stream_authorization so that web-public streams are
added in the list of authorized streams that a guest user can
subscribe.

This commit is part of PR #14638 that aims to allow guest users
to browse and subscribe to web-public streams.
2020-07-29 17:52:36 -07:00
Clara Dantas e22e12fe60 streams: Grant guest users access to web-public streams.
In this commit, we grant guest users access to stream history,
send message and common stream data of web-public streams.

This is part of PR #14638 that aims to allow guest users to
browse and subscribe to web-public streams.
2020-07-29 17:52:36 -07:00
Clara Dantas fda94a49c1 streams: Change flow in filter_stream_authorization.
This change makes the flow more coherent by instead of checking,
in the last condition, if the user isn't authorized to access that
stream, check if they are, as it is done in the other checks. Only
if all the conditions are false, which means that the user doesn't
have access to that stream, the stream is added to the
unauthorized_streams list.
2020-07-29 17:52:36 -07:00
arpit551 e0c9b0fdff audit_log: Log RealmAuditLog when Stream is created.
Added new Event Type in AbstractRealmAuditLog STREAM_CREATED.
Since we finally create streams in create_stream_if_needed function
in zerver/lib/streams.py so logged realm_audit there.
Passed acting_user when create_stream_if_needed or ensure_stream
function  is called.
Added tests in test_audit_log.
2020-07-06 17:32:11 -07:00
Mohit Gupta f8d1e0f86a refactor: Rename convert to markdown_convert.
Prior to this commit whenever convert was imported from zerver.lib.markdown
it was aliased as markdown_convert for readability.
This commit rename convert function to markdown_convert so that it can be
directly import it without aliasing and without compromising readability.
2020-07-06 12:39:59 -07:00
Mohit Gupta 66d54d86ad refactor: Rename remaining bugdown_convert to markdown_convert.
This commit is part of series of commits aimed at renaming bugdown to
markdown.
2020-06-29 15:03:20 -07:00
Mohit Gupta 3f5fc13491 refactor: Rename zerver.lib.bugdown to zerver.lib.markdown .
This commit is first of few commita which aim to change all the
bugdown references to markdown. This commits rename the files,
file path mentions and change the imports.
Variables and other references to bugdown will be renamed in susequent
commits.
2020-06-26 17:08:37 -07:00
sahil839 c488a35f10 streams: Add API for changing stream-level message_retention_days.
This commit adds backend support for setting message_retention_days
while creating streams and updating it for an existing stream. We only
allow organization owners to set/update it for a stream.

'message_retention_days' field for a stream existed previously also, but
there was no way to set it while creating streams or update it for an
exisiting streams using any endpoint.
2020-06-18 17:00:15 -07:00
Anders Kaseorg f364d06fb5 python: Convert percent formatting to .format for translated strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 16:24:46 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00
sahil839 46ef6816b6 stream: Allow non admins to set stream post policy when creating streams.
This commit allows non admins to set stream post policy while creating
streams.

Restriction was there to prevent user from creating a stream in which
the user cannot post himself but this will be taken care of with
stream admin feature.
2020-05-16 14:53:22 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Udit107710 16218d6de3 streams: Remove dependency of streams on actions.
Refactored code in actions.py and streams.py to move stream related
functions into streams.py and remove the dependency on actions.py.

validate_sender_can_write_to_stream function in actions.py was renamed
to access_stream_for_send_message in streams.py.
2020-04-18 16:56:59 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Stefan Weil d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
Ryan Rehman 174b2abcfd settings: Migrate to stream_post_policy structure.
This commit includes a new `stream_post_policy` setting,
by replacing the `is_announcement_only` field from the Stream model,
which is done by mirroring the structure of the existing
`create_stream_policy`.

It includes the necessary schema and database migrations to migrate
the is_announcement_only boolean field to stream_post_policy,
a smallPositiveInteger field similar to many other settings.

This change is done to allow organization administrators to restrict
new members from creating and posting to a stream. However, this does
not affect admins who are new members.

With many tweaks by tabbott to documentation under /help, etc.

Fixes #13616.
2020-02-04 17:08:08 -08:00
Mateusz Mandera 01288ede9e recipients: Remove bulk_get_recipients function and its uses.
With the recipient field being denormalized into the UserProfile and
Streams models, all current uses of bulk_get_recipients can be done more
efficient, by simply checking the .recipient_id attribute on the
appropriate objects.
2019-12-12 12:00:13 -08:00
Mateusz Mandera 467833a974 streams: Eliminate some unnecessary get_stream_recipient calls. 2019-12-09 15:24:35 -08:00
Mohit Gupta e5482adec0 search: Add streams:public to search entire history of public streams.
Add ability to search entire message history of all public streams at
once. It includes all subscibed, non subscribed public streams messages
and even historical public stream messages sent before user had joined
an organization or stream.

Fixes #8859.
2019-08-22 13:40:49 -07:00
Priyank Patel 3680393b47 messages: Support passing user ID for stream operator.
ok_to_include_history fuction was updated to expect stream ID.

Fixes part of #9474.
2019-08-17 11:19:12 -07:00
Priyank Patel 5eab5bbc3e streams: Refactor can_access_stream_history.
This refactor extracts the code logic of checking if user can access
stream history into it's own function: can_access_stream_history
that takes in user_profile and stream. Then we make seperate function
can_access_stream_by_name that takes in stream_name and retrives stream
and pass it to can_access_stream_history. This will make it easily to later
add a function that does the same thing with stream ID.
2019-08-17 11:10:00 -07:00
Priyank Patel 463ecb375f refactor: Add get_stream_by_narrow_operand_access_unchecked.
This let's us clean up the linter that excludes the use of get_stream
and by adding the access_unchecked in the name we make it clear that
it should be used with caution.

Refactoring idea by Tim Abbott.
2019-08-17 11:10:00 -07:00
Anders Kaseorg becef760bf cleanup: Delete leading newlines.
Previous cleanups (mostly the removals of Python __future__ imports)
were done in a way that introduced leading newlines.  Delete leading
newlines from all files, except static/assets/zulip-emoji/NOTICE,
which is a verbatim copy of the Apache 2.0 license.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-08-06 23:29:11 -07:00
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Anders Kaseorg f0ecb93515 zerver core: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:41:24 -08:00
Vaibhav 268c25e95f streams: Prevent members from creating admin-only streams.
This a check on server side to verify whether the user sending request
to create stream where only admins can post is an admin or not; Raises
a JsonableError when the user is not the realm admin.
2019-01-24 10:51:45 -08:00
Steve Howell c35afb942f Accept stream_id for muting endpoints. 2018-12-27 20:26:30 -08:00
Yashashvi Dave 6e136be975 default stream: Allows admins to remove any default stream.
This fixes a bug where administrators couldn't remove private
unsubscribed streams from the "default streams" list, because
access_stream_by_name didn't give them access to the stream object.
2018-08-07 13:28:30 -07:00
Aditya Bansal a68376e2ba zerver/lib: Change use of typing.Text to str. 2018-05-12 15:22:39 -07:00
Tim Abbott 8b26f912af streams: Limit access to public streams for guest users.
With most of the tests tests written by Shubham Dhama.
2018-05-04 09:47:58 -07:00
Aditya Bansal 75d76e4eb3 lib/streams.py: Extract get_stream_by_id as a separate function.
We extract get_stream_by_id function out of the body of
access_stream_by_id function to help us access streams for archives.
2018-05-02 15:23:33 -07:00
Tim Abbott bec71d7a50 messages: Add a server-level setting to control private stream history.
We don't indend for this server-level setting to exist in the long
term; the purpose of this is just to make it easy to test this code
path for development purposes.

This implements much of the Message side part of #2745.
2018-04-04 16:18:46 -07:00
Tim Abbott 228f41e916 messages: Pass UserProfile to is_public_stream_by_name and rename.
The new name can_access_stream_history_by_name gets to the point of
what this function actually does.  And passing in a user object lets
us define what this does based on the user subscribed.
2018-04-04 15:13:11 -07:00
neiljp (Neil Pilgrim) 2079c5e4d7 mypy: Mark Subscription as Optional in lib/streams.py:access_stream*. 2018-03-23 11:32:00 -07:00
YJDave 2031118545 stream settings: Allow realm admins to access all private stream subs.
This will allow realm admins to access subscribers of unsubscribed
private stream.  This is a preparatory commit for letting realm admins
remove those users.
2018-03-13 14:59:09 -07:00
YJDave 37f9d5c193 stream settings: Allow realm admins to update any stream name & description.
This will allow realm admins to update the names and descriptions of
private streams even if they are not subscribed, which fixes the buggy
behavior that previously nobody could(!).
2018-03-13 14:59:09 -07:00