Commit Graph

27583 Commits

Author SHA1 Message Date
Anders Kaseorg de5e3d3fbd tools/review: Don’t pretend to emulate shell=True either.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:08 -07:00
Anders Kaseorg 9a99d76296 tools: Remove pointless uses of shell=True.
Which, not coincidentally, is all of them.  Don’t use shell=True.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 22:39:06 -07:00
Tim Abbott e04156eef3 export: Fix error messages for stream list mismatches.
The previous error messages for this were written for a tool only to
be used by a couple people, and didn't make clear what potential
causes were.  Tweak these to provide greater clarity about what's
going on.

The main cause of these errors appearing in practice was fixed in
7ea5987e5d, but nothing strongly
prevents a similar issue from being introduced in the future.

Fixes #10078.
2018-07-30 22:32:26 -07:00
Tim Abbott 6317064210 unminify: Fix source map extraction for hash-named files.
Apparently, our old unminify logic relied on the fact that the
filenames displayed in tracebacks were of the form "app.js" (and the
`app.js` copy of the source map in the appropriate
`/home/zulip/deployments/`).  The correct behavior is to just look up
the source map for the appropriate hash-named
`app.a40806b10565c1dee5bf.js` type file.

We fix this with a few small tweaks to the regular expressions.  I
wish this file had reasonable unit tests.
2018-07-30 22:09:37 -07:00
Tim Abbott 7ea5987e5d errors: Use a setting to control the stream for slow-query logs.
We already had a setting for whether these logs were enabled; now it
also controls which stream the messages go to.

As part of this migration, we disable the feature in dev/production by
default; it's not useful for most environments.

Fixes the proximal data-export issue reported in #10078 (namely, a
stream with nobody ever subscribed to having been created).
2018-07-30 17:40:20 -07:00
Tim Abbott a8e5551395 restart-server: Ensure we restart process-fts-updates.
This is mostly important in that if you're running this as part of a
follow-up to a failed upgrade, and you don't do this,
process-fts-updates will be left not running, resulting in full-text
search not updating.
2018-07-30 16:27:53 -07:00
Tim Abbott 7853254df7 puppet: Run thumbor by default on voyager systems.
With this change, all that one needs to do to start using thumbor in
production is to set the `THUMBOR_URL` setting.

Since without THUMBOR_URL enabled, the thumbor service doesn't
actually do anything, this is pretty safe.
2018-07-30 16:16:52 -07:00
Shubham Padia bf6dc4472b models: Add is_private flag to UserMessage and add index for it.
The is_private flag is intended to be set if recipient type is
'private'(1) or 'huddle'(3), otherwise i.e if it is 'stream'(2), it
should be unset.

This commit adds a database index for the is_private flag (which we'll
need to use it). That index is used to reset the flag if it was
already set. The already set flags were due to a previous removal of
is_me_message flag for which the values were not cleared out.

For now, the is_private flag is always 0 since the really hard part of
this migration is clearing the unspecified previous state; future
commits will fully implement it actually doing something.

History: Migration rewritten significantly by tabbott to ensure it
runs in only 3 minutes on chat.zulip.org.  A key detail in making that
work was to ensure that we use the new index for the queries to find
rows to update (which currently requires the `order_by` and `limit`
clauses).
2018-07-30 15:43:55 -07:00
Shubham Padia 28589c5563 search pills: Backspace should remove a search pill with typeahead open.
Fixes part of #10026.

Typeaheads stopped propogation of keydown and keyup events for any
key except tab and enter. If stopAdvance was true even tab and enter
were not allowed.

advanceKeyCodes option was added to typeahead which allowed to specify
key codes for which propogation of keydown and keyup events should not
stop. advanceKeyCodes does not respect the stopAdvance option.
As the backspace key code is added to advanceKeyCodes in search.js,
the backspace key deletes pill on pressing backspace if input is empty
or only consists of spaces.
2018-07-30 14:33:06 -07:00
Shubham Dhama 4186ebe56a settings_user: Don't render bot_owner_select for user list.
This optimize the case when the user-info-form modal is opened
in user-list by not rendering bot_owner_select handlebar.
This bug is before changing form to modal.
2018-07-30 13:48:36 -07:00
Shubham Dhama 8f115c8291 settings_user: Refactor `open-user-form` click handler. 2018-07-30 13:48:36 -07:00
Shubham Dhama 5506f74084 settings_user: Remove redundant verbose check of owner value. 2018-07-30 13:48:36 -07:00
Shubham Dhama e08daf9a00 settings_user: Add modal for user info and roles form.
Advantages of changing it to modal is that now it doesn't render
form for every row i.e. form is rendered only for row edit button
is clicked.
2018-07-30 13:48:36 -07:00
Shubham Dhama 2b738d02c3 org settings: Refactor modal "partials" to a seperate file. 2018-07-30 13:48:36 -07:00
Shubham Dhama 2bcd21aeca org settings: Use people.get_active_human_persons to get active humans. 2018-07-30 13:48:36 -07:00
Shubham Dhama 16357fc84a people: Add common function for getting active human users. 2018-07-30 13:48:36 -07:00
Tim Abbott fd71f7282d docs: Clean up the new thumbnailing docs. 2018-07-30 13:46:32 -07:00
Tim Abbott b53463d6a4 docs: Remove use of multi-line 'su zulip' in docs.
It turns out this confused people into trying to run both pieces in
one line.
2018-07-30 13:38:55 -07:00
Tim Abbott aa5959396d docs: Add some basic subsystem documentation for thumbnailing. 2018-07-30 13:20:54 -07:00
Aditya Bansal 77651ece39 thumbnails: Rename size value 'original' to 'full'. 2018-07-30 13:00:23 -07:00
Aditya Bansal 5b5d8bb310 thumbnails: Rename data-original to data-src-fullsize. 2018-07-30 13:00:23 -07:00
Tim Abbott d49cee3050 manage.py: Catch issue with users running manage.py as the wrong user.
Apparently, if you tried running manage.py as a non-root user that
isn't the main zulip user, you'll get a confusing Django exception
about `SECRET_KEY` not being configured.

This change provides a clean, understandable exception for this case.
2018-07-30 12:34:48 -07:00
Tim Abbott 02ae71f27f api: Stop using API keys for Django->Tornado authentication.
As part of our effort to change the data model away from each user
having a single API key, we're eliminating the couple requests that
were made from Django to Tornado (as part of a /register or home
request) where we used the user's API key grabbed from the database
for authentication.

Instead, we use the (already existing) internal_notify_view
authentication mechanism, which uses the SHARED_SECRET setting for
security, for these requests, and just fetch the user object using
get_user_profile_by_id directly.

Tweaked by Yago to include the new /api/v1/events/internal endpoint in
the exempt_patterns list in test_helpers, since it's an endpoint we call
through Tornado. Also added a couple missing return type annotations.
2018-07-30 12:28:31 -07:00
Tim Abbott 07af59d4cc tornado: Split get_events_backend into two functions.
The lower-layer function, now called get_events_backend, is intended
to be called by multiple code paths (including the upcoming
get_events_internal).
2018-07-30 12:28:31 -07:00
Harshit Bansal 8aabce5a5f hotkeys: Use canonical name for reacting with `+` hotkey.
This is sort of a temporary fix to bring the state back to how it
was in commit: ef4337edcb. However,
long-term we will need to fix our local echo feature to do merging
of names just like we do on backend.
2018-07-30 12:18:33 -07:00
Tim Abbott 906718f154 node tests: Fix minor linter error.
I'm not sure how this slipped in.
2018-07-30 12:17:27 -07:00
Tim Abbott 00f6fc2559 subs: Limit editing subscribers for waiting period users.
Does not let you subscribe other users if you are a guest or a
waiting period user.

Co-authored-by: Shubham Padia <shubhamapadia@gmail.com>
2018-07-30 12:12:51 -07:00
Shubham Padia 1015b2fc44 subs: Refactor and add tests for UserProfile.can_create_streams.
This is a preparatory refactor for adding
UserProfile.can_subscribe_other_users.
Although there existed a test for limiting users from creating
streams at `test_subs.test_user_settings_for_adding_streams`,
it did not test the logic inside can_add_streams, tests have
been added to solve that issue.
2018-07-30 12:12:51 -07:00
Tim Abbott 30a3c48ff3 thumbor: Fix missing virtualenv-clone package in dependencies.
This package is important in order to avoid scary-looking errors
whenever we upgrade the dependencies in thumbor.txt (where
virtualenv-clone isn't installed in the venv, and then gets installed
by the code we just added a TODO comment to.
2018-07-30 11:55:16 -07:00
Anders Kaseorg a5407e1c7d scripts: Replace node-wrapper with a symlink.
Commit 00e057bf44 (#4727) simplified
node-wrapper to a one-line wrapper script for performance.  Copying
the binary was proposed and rejected because node finds some of its
modules relative to its own path.  But a symlink doesn’t have that
issue, as you can verify with

    node -e 'console.log(require.resolve.paths("foo"))'

(To find its own path, node uses `process.execPath`, which resolves
symlinks, and there’s no plausible reason for that behavior to change.
https://github.com/nodejs/node/blob/v8.11.1/lib/module.js#L708-L717
https://github.com/nodejs/node/blob/v10.7.0/lib/internal/modules/cjs/loader.js#L761-L770)

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-07-30 11:48:10 -07:00
Akash Nimare 1ce31ecd7d register: Update Zulip production server installation link.
Tweaked by tabbott to also recommend Zulip Cloud.
2018-07-30 11:43:09 -07:00
Akash Nimare e9e5a722c5 register: Update UI of realm_creation_fail page. 2018-07-30 11:41:07 -07:00
Steve Howell 6a4bacbd18 stream settings: Avoid redundant sorting step.
We don't need to get our original items in sorted
order, especially since we re-sort them using
a different comparison function.
2018-07-30 11:25:32 -07:00
Steve Howell f63dde8777 node tests: Add coverage to subs.js.
We test populate_stream_settings_left_panel() and use
its values to simulate data for filter_table().
2018-07-30 11:25:32 -07:00
Steve Howell 51edcb02c8 Streamline typeahead for stream links.
We don't need to get sorted streams in the "source"
function for typeahead, since we sort them later,
and we don't need to recalculate values.

This preserves the behavior that we include
unsubscribed streams in the typeahead, which is
probably intentional.
2018-07-30 11:25:32 -07:00
Steve Howell 132d0971ae Remove stream data stubs in typeahead tests.
We now use real stream_data functions, rather than
stubbing some of them to return strings instead of
structures.
2018-07-30 11:25:32 -07:00
Steve Howell 5c4d44bf3d refactor: Extract subs.get_stream_id_buckets().
This pulls the essential bucketing/sorting logic out
of filter_table().

The diff isn't quite as clean as I'd like, but some
of the code that got added back to filter_table() can be
eliminated in the future.  Basically, all the stuff
related to hidden ids can just be zapped if we go
to an approach of just re-building the DOM cleanly
whenever our filters change.
2018-07-30 11:25:32 -07:00
Steve Howell 68dba4515d refactor: Use triage_stream() to filter streams.
We replace two calls to stream_matches_query() with
a single call to triage_stream(), which prevents us
from doing the same is-subscribed checks twice.
2018-07-30 11:25:32 -07:00
Steve Howell 95edf68fd0 Refactor stream_matches_query to early-exit.
For the non-subscribed case, checking flags is a quick
way to reject streams vs. the more expensive indexOf
checks.
2018-07-30 11:25:32 -07:00
Steve Howell 95d136ca5e Extract search_util.js module.
We probably should have done this a while ago, even
though these functions are pretty tiny.  The goal here
is to make it easier to have more consistent search
semantics.

Our first use case is subs.js.  In this case we
are able to decouple a bit of generic string
matching from the subs-specific code.
2018-07-30 11:25:32 -07:00
Steve Howell 064d0f3c89 Extract stream_data.sort_for_stream_settings().
We move some data code from subs.js to stream_data.js.

It's not clear we have been using the optimal sort for
dealing with locales, but this change preserves the
current behavior.  The only subtle change here is that
we look up subs using a Dict now instead of a plain
JS object.
2018-07-30 11:25:32 -07:00
Steve Howell 5641c77c94 refactor: Introduce other_stream_ids in filter_table().
We now build up three buckets of stream ids for slightly
more consistent code that will help in future refactorings.
2018-07-30 11:25:32 -07:00
Vishnu Ks 32438cde9f version: Bump PROVISION_VERSION for py packages upgrade. 2018-07-30 10:09:19 -07:00
Vishnu Ks 6f17db5731 requirements: Upgrade pytz to 2018.5.
https://github.com/stub42/pytz/blob/master/tz/NEWS
2018-07-30 10:09:19 -07:00
Vishnu Ks 2b57e1f639 requirements: Upgrade wheel to 0.31.1. 2018-07-30 10:09:18 -07:00
Vishnu Ks 9e39e136b9 requirements: Upgrade twilio to 6.15.2.
Lot of bug fixes and changes. But nothing looks like
it will bfeak Zulip.
2018-07-30 10:06:23 -07:00
Vishnu Ks de77728c86 requirements: Upgrade SQLAlchemy to 1.2.10.
Bunch of bug fixes.
http://docs.sqlalchemy.org/en/latest/changelog/
changelog_12.html#change-1.2.10
2018-07-30 10:06:23 -07:00
Vishnu Ks 8b857db5ae requirements: Upgrade sphinx-rtd-theme to 0.4.1. 2018-07-30 10:06:23 -07:00
Vishnu Ks ec5fad4cac requirements: Upgrade Sphinx to 1.7.6. 2018-07-30 10:06:23 -07:00
Vishnu Ks 46338f5418 requirements: Upgrade setuptools to 40.0.0. 2018-07-30 10:06:23 -07:00