Commit Graph

29044 Commits

Author SHA1 Message Date
Steve Howell 876a72c467 hipchat import: Extract get_hipchat_sender_id(). 2018-10-23 17:27:37 -05:00
Steve Howell 481488a35e Extract make_subscriber_map().
We extract this function and put it in the shared
library `import_util.py`.

Also, we make it one time higher up in the call
stack, rather than re-building it for every batch
of messages.  I doubt this was super expensive, but
there's no reason to repeatedly execute this.
2018-10-23 17:27:37 -05:00
Steve Howell 737e02a2e6 hipchat import: Fix PM messages.
Before this fix, we were creating two copies of every
PM Message in zerver_message with only corresponding
UserMessage row.

Now we only create one PM Message per message, which
we accomplish by making sure we only use imported
messages from the sender's history.json file.  And
then we write UserMessage rows for both participants
by making sure to include sender_id in the set of
user_ids that feeds into making UserMessage.  For
the case where you PM yourself, there's just one
UserMessage row.

It does not appear that we need to support huddles
yet.
2018-10-23 17:27:37 -05:00
Steve Howell 53436b4b41 import: Rename id_maps -> ID_MAP. 2018-10-23 17:27:37 -05:00
Steve Howell bd9e4ef0c8 import: Use pub_date to sort message ids.
When we create new ids for message rows, we
now sort the new ids by their corresponding
pub_date values in the rows.

This takes a sizable chunk of memory.

This feature only gets turned on if you
set sort_by_date to True in realm.json.
2018-10-23 17:27:37 -05:00
Steve Howell d1ff903534 refactor: Rename build_user -> build_user_profile.
This makes greps less confusing.
2018-10-23 17:27:37 -05:00
Rishi Gupta b26e8eca62 portico: Redirect /plans to zulipchat.com/plans for self-hosted realms. 2018-10-23 14:47:00 -07:00
Rishi Gupta fbd884f934 billing: Rename MAX_INVITES_PREMIUM.
Largely to have parallel structure with INVITES_DEFAULT_REALM_DAILY_MAX.
2018-10-23 14:47:00 -07:00
Rishi Gupta a7fe6b1c34 billing: Fix max_invites for PREMIUM_FREE.
We could migrate all the current PREMIUM_FREE organizations to have more
invites, but this setting mainly affects orgs right as they are starting, so
it's probably fine.
2018-10-23 14:47:00 -07:00
Eeshan Garg 4c0890e8b0 webhooks/github: Handle empty 'requested_reviewers' key.
We recently received a bug report that implied that for certain
payloads, the `requested_reviewers` key was empty whereas a
singular `requested_reviewer` key containing one reviewer's
information was present in its stead. Naturally, this raised
some not so pretty IndexError exceptions.

After some investigation and generating a few similar payloads,
I discovered that in every case both the `requested_reviewers`
and the `requested_reviewer` keys were correctly populated, so I
had to manually edit the payload to reproduce the error on my end.

My guess is that this anomaly goes back to when GitHub's reviewer
request feature was new and didn't support requesting multiple
reviewers, and that the singular `requested_reviewer` key could
possibly just be there for backwards compatibility or might just
be mere oversight. Either way, the solution here is to look for the
plural `requested_reviewers` key, and if that is empty, fall back
to the singular `requested_reviewer` key.
2018-10-23 14:41:24 -07:00
Shubham Padia 5ea7feee7e upgrade-zulip-from-git: Support specifying tag or commit ID for refname.
Fixes #10706.
Issue: Before this commit, the `refname` positional argument to
`upgrade-zulip-from-git` script would run successfully for a branch
name on the given remote, but the script would fail if it was
provided with a tag or commit ID.
Solution: 'git clone -q -b refname LOCAL_GIT_CACHE_DIR deploy_path`
would be split into two commands:
1.) `git clone -q LOCAL_GIT_CACHE_DIR deploy_path`
2.) `git checkout -b deploy_timestamp refname` which makes a new
branch with the same name as the timestamp used in make_deploy_path.
2018-10-23 11:11:08 -07:00
neiljp (Neil Pilgrim) 9f8d778d20 integrations: Update Matrix and IRC docs.
[with edits by Rishi Gupta]
2018-10-22 18:04:22 -07:00
Rishi Gupta 1c264dedc5 api docs: Change Shakespeare quote.
The previous quote doesn't come off well if you don't know the reference
(which the majority of our users will not).
2018-10-22 15:44:36 -07:00
Scott Lee b4f95aa124 doc: Expand documentation for SSH port forwarding.
This adds additional links and examples to make it much more clear how
to do secure SSH port-forwarding with a remote development environment.
2018-10-22 14:53:52 -07:00
Tim Abbott d69d71c59f i18n: Sync translation data from transifex. 2018-10-22 14:50:46 -07:00
Tim Abbott adf918cf5a upgrade-zulip-from-git: Fix linter error from too-long line. 2018-10-22 14:11:40 -07:00
Eeshan Garg 3a47a1a627 webhooks/greenhouse: Ignore ping events.
"ping" type events are usually just Greenhouse's way of indicating
that a webhook has been set up.
2018-10-22 12:27:48 -07:00
Steve Howell c58a99b156 PM sidebar: Expand PM sidebar for huddles.
This fixes a regression from here:

    88b4a9f2d7

The fix didn't account for how huddles are
represented as comma-delimited strings.

We also simplify the logic by extracting a
function and doing early-exit for simple
cases.
2018-10-22 12:22:26 -07:00
Steve Howell 328e2ff316 Fix "Copy link to conversation" links.
This cleans up the code for stream links
and creates nicer, more correct links for
PMs.

Fixes #10605
2018-10-22 12:22:26 -07:00
Steve Howell adf616d3f1 Add pm_perma_link helper. 2018-10-22 12:22:26 -07:00
Steve Howell 8c49985058 url decoding: Handle your own id being in PM slugs.
Internally we generally omit our own id and email
in data structures related to PMs, except when we
are the sender, but if we receive "perma links"
we will need to filter out our id.
2018-10-22 12:22:26 -07:00
Scott Lee 32f1a13c7b docs: Add note about creating a non-root user.
Tweaked by tabbott to move most of the content into the remote
development documentation, both for brevity in the main docs as well
as clarity.

Fixes #10694.
2018-10-22 12:19:00 -07:00
Shubham Padia bdb5f43370 upgrade-zulip-from-git: Support specifying remote on the command line.
Adds an optional argument `--remote-url` to specify the remote URL.
Command line remote URL will be given preference above the one
in /etc/zulip/zulip.conf.

Fixes #6092.
2018-10-22 12:01:05 -07:00
Shubham Padia 094e50753e upgrade-zulip-from-git: Rename git_url variable to remote_url. 2018-10-22 12:00:40 -07:00
Shubham Padia bba01e210b upgrade-zulip-from-git: Refactor to use argparse instead of sys.argv.
This is a preparatory commit to add an optional argument to mention
the git remote url. This does not make any change to the current
behaviour.
2018-10-22 12:00:39 -07:00
Aditya Bansal a16bf34c7f thumbnailing: Fix oversharpening of thumbnails.
We seemed to have been doing too much of sharpening on the thumbnails.
The purpose of sharpening here was to just counter the softening
effects of a resize on an image but overdoing it is bad.

Value sharpen(0.5,0.2,true) seems to look good for achieving the
best results here on different displays as revealed in the manual
hit and trial based testing.

Thanks to @borisyankov for pointing out the issue and suggesting
the values.
2018-10-22 22:28:04 +05:30
Vishnu Ks 3b026559d4 stripe: Make stripe.error stubs more similiar to stripe python lib.
This will also fix the error that is generated during
the mypy 0.641 upgrade in PR #10691.
2018-10-22 13:51:05 +05:30
Rishi Gupta c37384f63d api docs: Update writing-bots.
Removes the intro material since there are better places for it.
2018-10-20 13:31:19 -07:00
Rishi Gupta 6b8de50c50 api docs: Update running-bots.
This page still has some work needed, but will probably require coordination
with the product work as well.
2018-10-20 13:31:19 -07:00
Rishi Gupta 796a1a6582 api docs: Update non-webhook-integrations.md.
Addresses comments on a8e1225.
2018-10-20 13:31:19 -07:00
Rishi Gupta e0c6f62618 user docs: Update stream-permissions to note message access via bots. 2018-10-20 13:31:19 -07:00
Pragati Agrawal b665b0a9a5 users: Avoid O(N) query checking for is_admin.
The main benefit is that we don't do O(N) queries for checking admin
thus leading to less processing of UserProfile.
2018-10-19 15:51:12 -07:00
Pragati Agrawal cc0ac4d340 settings_users: Change "role" column text on updation.
This reflects the newly selected value of role in "role" column under
active-users section and deletes the redundant admin-icon updation code(
As we already removed bolt admin-icon)
2018-10-19 15:51:12 -07:00
Rishi Gupta b60005c5bb history: Add NSF and GSoC logos in a new support section. 2018-10-19 15:50:09 -07:00
Aditya Ramesh c242433dcb message view: Fix order of dynamic elements in recipient bar.
This swaps the order of the "blue square" outgoing link and the edit
pencil.

Fixes #10681.
2018-10-19 15:42:47 -07:00
Rishi Gupta 39c56b3ba9 docs: Update security-model.md.
Tweaked by tabbott to fix a few typos and purge use of "realm".
2018-10-19 15:40:36 -07:00
Tim Abbott 39ea471cf1 error reports: Ensure we filter API keys from query strings.
For some webhook endpoints where the third-party API requires us to do
this, the user's API key might appear in error emails through
appearing in the `QUERY_STRING` parameter.  Fix that by filtering any
actual content from those; what we usually need for debugging is just
what set of parameters were provided.
2018-10-19 15:03:14 -07:00
Yashashvi Dave cc810f8951 settings: Fix width not maximum while uploading user avatar.
New user avatar width is not maximum when user upload
new image. Because wrong html element is accessed for
setting value of image src attribute.
This commit removes these code from success of ajax call,
cause we already handle this in event `user_events - avatar_url`.
2018-10-19 12:36:18 -07:00
Yashashvi Dave 41fbb16cdf org settings: Fix error when admin update realm users full names.
Currently, if there is only one admin in realm and admin tries
to updates any non-adminuser's full name it throws error,
"Cannot remove only realm admin". Because in `/json/users/<user_id>`
api check_if_last_admin_is_changed is checked even if property
is_admin is not changed.

This commit fix this issue and add tests for it.
2018-10-19 12:34:20 -07:00
Rishi Gupta 97ed71ca69 portico: Reorder quotes in carousel on /hello. 2018-10-18 16:31:03 -07:00
Rishi Gupta aad5c3df48 user docs: Fix typo causing broken gear icon. 2018-10-18 16:30:42 -07:00
Rishi Gupta f14ed03707 docs: Update import instructions in export-and-import. 2018-10-18 16:22:31 -07:00
Tim Abbott c3c83d7e83 docs: Document PROVISION_VERSION. 2018-10-18 15:38:55 -07:00
Anders Kaseorg c3bd293eaf Add shellcheck as a linter.
Fixes #10162.

Tweaked by tabbott to cover documentation as well.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:32:26 -07:00
Anders Kaseorg d0fb34e8af install: Fix shellcheck warnings.
In scripts/lib/install line 71:
ZULIP_PATH="$(readlink -f $(dirname $0)/../..)"
                          ^-- SC2046: Quote this to prevent word splitting.
                                    ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 105:
mem_kb=$(cat /proc/meminfo | head -n1 | awk '{print $2}')
             ^-- SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.

In scripts/lib/install line 141:
apt-get -y dist-upgrade $APT_OPTIONS
                        ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 145:
    $ADDITIONAL_PACKAGES
    ^-- SC2086: Double quote to prevent globbing and word splitting.

In scripts/lib/install line 254:
    if [ -n "ZULIP_ADMINISTRATOR" ]; then
             ^-- SC2157: Argument to -n is always true due to literal strings.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Anders Kaseorg 49ae9c1e44 pre-commit: Fix shellcheck warnings.
In tools/pre-commit line 18:
if [ -z "$VIRTUAL_ENV" ] && `which vagrant > /dev/null` && [ -e .vagrant ]; then
                            ^-- SC2092: Remove backticks to avoid executing output.
                            ^-- SC2006: Use $(..) instead of legacy `..`.
                             ^-- SC2230: which is non-standard. Use builtin 'command -v' instead.

In tools/pre-commit line 23:
    ./tools/lint --no-gitlint --force $changed_files || true
                                      ^-- SC2086: Double quote to prevent globbing and word splitting.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2018-10-18 15:26:21 -07:00
Tim Abbott 695d8d0bd1 get_apns_payload: Require a UserProfile object for the recipient.
This is preparatory work for being able to display an unread count
badge on iOS, in which case we need to know who the current user is.
2018-10-18 15:09:18 -07:00
Tim Abbott a4df001cef check_queue_worker_errors: Add support for running unprivileged.
Previously, this script needed access to Django settings, which in
turn required access to /etc/zulip/zulip-secrets.conf.  Since that
isn't world-readable, this meant that this couldn't run as an
unprivileged `nagios` user.

Fix that by just hardcoding the appropriate path under /var/log/.
2018-10-18 15:03:17 -07:00
Rishi Gupta 690705858a billing: Update test_upgrade_where_..._fails_at_first to use @mock_stripe. 2018-10-18 11:16:00 -07:00
Rishi Gupta 03d77bc6af billing: Replace instance.save() Stripe API calls with class.save(instance).
This will make it easier to mock the calls in our new stripe mocking
framework. I believe the two forms are equivalent, assuming the Stripe
Python bindings aren't doing anything crazy. And if not, well hopefully our
new testing framework will catch it :).

[Idea originally from Vishnu KS.]
2018-10-18 11:16:00 -07:00