Commit Graph

40317 Commits

Author SHA1 Message Date
Steve Howell c2e3d626e2 node tests: Move data setup to module level for peer_data.
We move some of the data setup to the top of the file.

We also remove some get_sub() calls that aren't really
necessary now that peer_data and stream_data are more
independent.
2021-02-03 15:23:17 -08:00
Steve Howell 17ea215f18 minor: Remove peer_data.clear(). 2021-02-03 15:23:17 -08:00
Steve Howell 58855e8224 refactor: Remove maybe_clear_subscribers().
The maybe_clear_subscribers() function was an artifact of
when we used to attach subscribers to the "sub" records in
stream_data.js.  I think it was basically a refactoring
shim, and due to some other recent cleanup, it was only
used in test code.

We also change how we validate stream ids.

Going forward, peer_data just looks up stream_ids with the
normal stream_data API when it's trying to warn about
rogue stream_ids coming in.  As I alluded to in an earlier
commit, some of the warning code here might be overly
defensive, but at least it's pretty self-contained.
2021-02-03 15:23:17 -08:00
Steve Howell e44e48ef20 minor: Add get_user_set call that I missed earlier.
In my recent commit to introduce get_user_set() I
inadvertently skipped one place to call it.

I also remove a return statement that was made
unnecessary by the new get_user_set() helper.
2021-02-03 15:23:17 -08:00
Steve Howell 016038dcd1 blueslip: Add measure_time wrapper.
Now when we want to measure how long a block
of code takes to execute, we just wrap it with
`blueslip.measure_time`, instead of the awkward
idiom from my original commit of getting a callback
function.

My rationale for the original scheme was that I
wanted to minimize diffs and avoid changing
`const` to `let` in a few cases, but I believe
now that the function wrapper is nicer.

In a few cases I just removed the blueslip timing
code, since I was able to confirm on czo that
the times were pretty minimal.
2021-02-03 15:23:17 -08:00
Anders Kaseorg 4bcf7131c1 push-to-pull-request: Wait for GitHub background update before merge.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-03 14:55:08 -08:00
aditya-mitra b8f57fa6bb provision: Add provision support for KDE Neon 20.04.
PostgreSQL 12 is used for Neon 20.04, which is a fork of Ubuntu 20.04.
2021-02-03 11:51:05 -08:00
Anders Kaseorg 4c09a549c9 release-checklist: Push dev tags to zulip-internal.git too.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-02-03 10:07:47 -08:00
Aman Agrawal 58a4469896 version: Correct API feature level.
API feature level should have been increased along with #17010
which increased the community topic editing time.
2021-02-03 10:20:59 -05:00
Aditya Mitra aafd9ee2b6
docs: Fix typo in setup-advanced.md.
Fixed the grammatical error by changing long to longer
2021-02-01 08:58:32 -08:00
Ganesh Pawar bc6c83c12e docs: Remove duplicate entry of file in widgets documentation. 2021-01-30 09:01:13 -08:00
Ganesh Pawar f70629ea6c composebox: Add typeahead for /todo. 2021-01-30 09:01:13 -08:00
Ganesh Pawar 632abd45eb ui_report: Migrate ui_report.message to ui_report.client_error. 2021-01-30 07:48:44 -08:00
Ganesh Pawar 5802bbfb98 ui_report: Add ui_report.client_error. 2021-01-30 07:48:44 -08:00
Steve Howell e243af531b refactor: Extract get_user_set in peer_data.
We now use the same code in all places to
get the bucket of user_ids that correspond
to a stream, and we consistently treat
a stream as having zero subscribers, not
an undefined number of subscribers, in
the hypothetical case of us asking about
a stream that we're not tracking.

The behavior for untracked streams has
always been problematic, since if a
stream is untracked, all bets are off.

So now if we don't "track" the stream,
the subscriber count is zero.  None of
our callers distinguish between undefined
and zero.

And we just consider the stream to be subscribed
by a user when add_subscriber is called,
even if we haven't been told by stream_data
to track the stream.  (We also stop
returning true/false from add_subscriber,
since only test code was looking at it.)

We protect against the most likely source
of internal-to-the-frontend bugs by adding
the assert_number() call.

We generally have to assume that the server
is sending us sensible data at page load
time, or all bets are off.

And we have good protections in place
for unknown ids in our dispatch code
for peer_add/peer_remove events.
2021-01-29 15:21:07 -08:00
Steve Howell 52281462e2 zephyr: Close exemption for stream-link warnings.
For the rare case where you're doing a link to a private
stream from a larger private stream that is a superset of
the former, we have bypassed warnings that you are linking
to a private stream.

I'm not sure we need this exemption for any situation
(just let the user bypass the warning), but we definitely
don't want false positives for the exemption.

For now I am closing down this loophole specifically for
Zephyr users.

Zephyr users are special in that we might not get
subscriber info on certain streams.

The current behavior for this edge case is a little
unclear.  The current implementation of
peer_data.is_subscriber_subset returns false if both
streams are untracked, but most streams are tracked if we
have a sub for them and just get treated as having an
empty set of subscribers.  And an empty set is a subset of
itself.  Upcoming changes to our server data are gonna
make this edge case even more annoying to maintain.
2021-01-29 15:21:07 -08:00
Steve Howell 5a27ed6a72 refactor: Change where we initialize scrolling code.
This is a more sensible place to initialize the
the scrolling handler, plus it breaks an annoying
indirect dependency on jQuery for the unit tests.
2021-01-29 15:21:07 -08:00
Steve Howell 6c4b1183f2 node tests: Move peer_data tests to new peer_data.js. 2021-01-29 15:21:07 -08:00
Steve Howell 2edfdb4ff8 refactor: Extract bulk functions to add/remove peers.
We also streamline some of the error handling code
by doing everything up front.  This will prevent
scenarios where a single bad stream_id/user_id causes a
bunch of the same warnings in an inner loop.
2021-01-29 15:21:07 -08:00
Steve Howell e820c43b77 node tests: Verify peer_add/peer_remove save changes. 2021-01-29 15:21:07 -08:00
Steve Howell 805a2b6670 refactor: Remove stream_data.update_subscribers_count.
This removes a bit of complexity.  If a piece of
settings code needs to render a stream with
subscribers, it just asks for it.

We no longer have the brittle, action-at-a-distance
mechanism of mutating the subscriber count on to
the stream_data version of a sub.

Stream subs are pretty small, so making copies of
them is cheap, and the blueslip timings from the
previous commit can help confirm that.

There is some discussion of putting `subscriber_count`
on the Stream model, which may eventually get us
away from tracking it in `peer_data.js`, but we will
cross that bridge when we get there.  See
https://github.com/zulip/zulip/issues/17101 for
more details.
2021-01-29 15:21:07 -08:00
Steve Howell 5bd68b5180 minor: Get blueslip times for stream settings. 2021-01-29 15:21:07 -08:00
Steve Howell dc9b4de05f refactor: Call peer_data.get_subscriber_count().
We are trying to move away from having the
subscriber count on the sub.

This commit handles the easy one-liners
in the JS code.
2021-01-29 15:21:07 -08:00
Steve Howell 5babe54310 typeahead: Use stream_weekly_traffic for tiebreaker.
The weekly stream traffic is a better tiebreaker
for stream typeaheads than subscriber count, as
it's more directly a measure of a stream's current
relevance.

Normally stream traffic and subscriber counts are
closely correlated, but a good example for me is
the #twitter feed on czo, which only has 80 subscribers,
but which gets more traffic than our #integrations
stream (with 16k subscribers).  I would rather
see #twitter win the tiebreaker (if it even got
to the tiebreaker).

The main motivation behind this fix, though, is
to break our dependency on peer_data, which has
some upcoming changes that will introduce some
performance tradeoffs, and I want one less place
to audit.

Also, it will be easier long term to share this
code with mobile if we don't require mobile
to pull in our peer_data dependency.  (The webapp
has different forces than mobile that dicate
our data structures.)
2021-01-29 15:21:07 -08:00
Nikhil Maske aadd42f9e8 design: Adjust the visibility of the spinner icon.
The spinner icon is not visible until the user clicks on topic_edit_save,
so the space alloted to spinner-icon looks empty for rest of the time.
To improve the design, the spinner icon is only shown when the user
clicks on topic_edit_save.
2021-01-29 14:55:06 -08:00
Vishnu KS 03879eec6d email_log: Mark EmailLogBackEnd.log_email as a staticmethod.
We don't access the self attribute in this method.
2021-01-29 14:51:38 -08:00
Vishnu KS 5870de0fa0 dev_settings: Remove email forwarding related settings.
These are now set in dev-secrets.
2021-01-29 14:51:38 -08:00
Vishnu KS 7d8ade3b2a dev_settings: Allow setting the value of EMAIL_USE_TLS from dev-secrets. 2021-01-29 14:51:38 -08:00
Vishnu KS edac24acf1 email_log: Inherit EmailLogBackEnd from smtp.EmailBackend.
EmailLogBackend used to create a new EmailMessage and copy
only certain values from the original EmailMultiAlternatives
object. This resulted in the loss of information and made
it harder to test PRs like
https://github.com/zulip/zulip/pull/17121.

So instead of creating a new EmailMessage, tweak and send the existing
EmailMultiAlternatives object.
2021-01-29 14:51:38 -08:00
Anders Kaseorg 049d140557 requirements: Upgrade moto to 1.3.17.dev230.
This unblocks the upgrade to responses 0.12.1.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-29 12:02:43 -08:00
Aman Agrawal b26727ed16 invite-new-users: Specify that the limit spans for the whole day. 2021-01-29 09:51:11 -08:00
Rohitt Vashishtha 75a0fa5b91 Rename list_render -> ListWidget.
Similar to DropdownListWidget, list_render is actually a widget. The
changed case more accurately represents how its supposed to be used
as a Class.
2021-01-29 09:36:35 -08:00
Anders Kaseorg 69890f36b1 migrations: Enforce evaluation order in 0306 WHERE clause.
Depending on PostgreSQL’s query plan, it was possible for the value
condition to be evaluated before the field_type condition was checked,
leading to errors like

psycopg2.errors.InvalidDatetimeFormat: invalid value "stri" for "YYYY"
DETAIL:  Value must be an integer.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-29 09:35:52 -08:00
Ganesh Pawar a42f7a67e1 populate_db: Add images in test data.
This isn't quite the right model, because we're not actually going
through the upload code path, but it does at least provide some inline
image previews in the data.

Fixes part of #14991.
2021-01-27 17:52:28 -08:00
Anders Kaseorg 4ca66e7278 timezone: Correct common_timezones dictionary.
The changes are as follows:

• Fix one day offset in all western zones.
• Correct CST from -64800 to -21600 and CDT from -68400 to -18000.
• Disambiguate PST in favor of -28000 over +28000.
• Add GMT, UTC, WET, previously excluded for being at offset 0.
• Add ACDT, AEDT, AKST, MET, MSK, NST, NZDT, PKT, which the previous
  code did not find.
• Remove numbered abbreviations -12, …, +14, which are unnecessary.
• Remove MSD and PKST, which are no longer used.

Hardcode the dict and verify it with a test, so that future
discrepancies won’t go silently unnoticed.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-27 15:23:15 -08:00
Tim Abbott fd8504e06b munin: Update to use NAGIOS_BOT_HOST.
We haven't actively used this plugin in years, and so it was never
converted from the 2014-era monitoring to detect the hostname.

This seems worth fixing since we may want to migrate this logic to a
more modern monitoring system, and it's helpful to have it correct.
2021-01-27 12:07:09 -08:00
Riken Shah baae227efb run-dev: Fix dev url showing extra port.
This commit fixes the issue of `run-dev.py`
showing the development URL with an
extra port when EXTERNAL_HOST
is specified.

Fixes: #17054
2021-01-27 11:37:26 -08:00
Anders Kaseorg 1a148f5d9a scroll_bar: Fix incorrectly converted breakpoint.
Commit e941ee4a15 (#16680) incorrectly
converted this from 775px to xl-max = 1199px instead of md-max =
767px, causing misplacement of the FRB for browser widths between
these values.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-27 10:32:45 -08:00
Aman Agrawal 8729965548 last_seen_status_from_date: Fix incorrect last seen value.
We use day_old calculated based on day instead of hours to
render last seen values. This fixes us incorrectly quoting
anything 24 - 48 hours ago as Yesterday and
incorrectly quoting `time` that are Yesterday
but < 24 hours ago in 'x hours ago' format.
2021-01-27 09:58:22 -08:00
Aman Agrawal de4aeacbb1 left-sidebar: Only add expanded class to it if it is a popover.
We were adding `expanded` class to left-sidebar when searching
for streams even if the left-sidebar was not in the popover state.
This cased confusion with popovers.any_active returning true,
when actually it is not.
2021-01-27 09:58:22 -08:00
Aman Agrawal 459e1fc218 sent_messages: Pass correct reference of this in setTimeout.
We explicitly bind `this` to MessageState class which otherwise
was defaulting to `window`.

This resulted in variables like `this.received` and `this.local_id`
being incorrectly interpreted by called function
as `window.(received | local_id)` which are `undefined`.
Hence, frontend thinks that the message was never received.

It was noticed since this was the common log message when
a double message send bug was observed. This change in no
was indicates fixing of the double send bug, but is hopefully
one step forward in that direction.
2021-01-27 09:52:42 -08:00
Anders Kaseorg d33217f2ef requirements: Remove unused httplib2 requirement.
It’s unused since commit 8dd95bd057
(#13718).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg e1815f5d11 requirements: Use Django extra to pull in argon2-cffi.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg 47674c2298 requirements: Use django-two-factor-auth extra to pull in twilio.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 16:33:32 -08:00
Anders Kaseorg a7bd1f8049 requirements: Upgrade Python requirements. 2021-01-26 13:27:50 -08:00
Anders Kaseorg c0ad595855 email_notifications: Fix HTML injection bug.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:50 -08:00
Anders Kaseorg 454144c35f queue_processors: Fix retry_send_email_failures type.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:50 -08:00
Anders Kaseorg 70aa9903b9 list_realms: Convert percent formatting to "".format.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:22 -08:00
Anders Kaseorg c36a66cc1b redis_utils: Convert percent formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:27:22 -08:00
Anders Kaseorg 948f2ee2ad manage: Quote commands correctly in log_management_command.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2021-01-26 13:26:57 -08:00