Commit Graph

23972 Commits

Author SHA1 Message Date
Greg Price 8fd98133d8 docs: Tweak discussion of SSL certs in prod requirements. 2018-01-23 10:57:50 -08:00
Greg Price 0cf0c51cad docs: Revise description of main install step.
Bulleted information instead of prose, huzzah!  Also I think we need
to explain the options a bit right here, or at least link to where
they're documented somewhere.  (If the list gets much longer, we'll
want to shift toward the latter.)

Also reorganize existing information a bit, and clean up a couple
of nits.
2018-01-23 10:57:50 -08:00
Greg Price bcc1456f55 docs: Tighten sub-headings in SSL certs doc.
What I really want is to give these sections nice stable slugs
to put on the anchors and use as the URL fragment, independent of
any wording tweaks on the text headings.  But I don't think we
have that feature with Markdown and our current docs infrastructure.

At least for Certbot, the brevity helps make this heading clearer
than the previous one.
2018-01-23 10:57:50 -08:00
Greg Price 93b957eaae docs: Revise explanation of manual SSL cert install.
Change a bit of prose to bullets.  Also lead with the "If you ..."
that helps readers skim past this section.
2018-01-23 10:57:50 -08:00
Tim Abbott 21d8ff14b4 docs: Update documentation to recommend certbot. 2018-01-23 10:57:50 -08:00
Tim Abbott 2365b13b68 puppet: Move postgres Nagios plugin to main postgres-common.
This plugins package is required in order to use Nagios checks to
verify the Zulip postgres database, and thus belongs in the default
package set.
2018-01-23 10:31:48 -08:00
Dennis Ludl 895a675f4c decorators: Fix email check in access_user_by_api_key case insensitive.
In Zulip, email addresses should always be treated as
case-insensitive; this code path incorrectly assumed the email input
by the user had the correct case.

Discussed in:

https://chat.zulip.org/#narrow/stream/issues/subject/Mobile.20Apps.3A.20Sometimes.20don't.20load.20data/near/461062

Commit message tweaked by tabbott.
2018-01-23 10:04:03 -08:00
Patrick Grave a2ed76c383 mypy: Use Python 3 syntax for typing in beanstalk/tests.py 2018-01-23 08:49:42 -08:00
Patrick Grave f2b322c32d mypy: Use Python 3 syntax for typing in upload.py 2018-01-23 08:49:42 -08:00
Patrick Grave 008e250668 mypy: Use Python 3 syntax for typing in test_runner.py 2018-01-23 08:49:42 -08:00
Abijith10 7ed76eeb23 mypy: Use Python 3 type syntax in zerver/webhooks/zendesk/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 c4420ba0c6 mypy: Use Python 3 type syntax in zerver/webhooks/librato/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 8a0cd1a966 mypy: Use Python 3 type syntax in zerver/webhooks/github_webhook/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 d30c2463c2 mypy: Use Python 3 type syntax in zerver/webhooks/deskdotcom/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 c6804f5c8d mypy: Use Python 3 type syntax in zerver/webhooks/delighted/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 39fe64132d mypy: Use Python 3 type syntax in zerver/webhooks/codeship/view.py. 2018-01-23 08:37:56 -08:00
Abijith10 ead63c475b mypy: Use Python 3 type syntax in zerver/models.py. 2018-01-23 08:37:56 -08:00
rht 6b4f392ee9 Slack importer: Make it transparent to user which parts aren't mapped 1-to-1. 2018-01-23 10:36:14 -05:00
Rhea Parekh 11b549e566 slack importer: Add regex for mail links.
`<mailto:foo@foo.com>` is changed to `mailto:foo@foo.com`.
2018-01-23 10:01:15 -05:00
Rhea Parekh df1bb5fd27 slack importer: Fix link regex and add test case. 2018-01-23 10:01:15 -05:00
Rhea Parekh 5b1e9f8181 slack importer: correct the implementation of unzipping slack data file.
Slack zip file unzips in the same folder.
2018-01-23 10:01:15 -05:00
Rhea Parekh 845a8d63bf slack importer: Prevent id clashes for import in an active db.
Previously we had a problem of id clashes while importing converted
slack data into an existing zulip instance with realms which are actively
populating the database.

This counts the total objects to be imported and does a db transaction
to increase the SEQUENCE number for that table by that number,
and hence allocates a range of ids for the to be converted slack data
objects.
2018-01-23 10:01:15 -05:00
Weronika Grzybowska 7ac7100a1d messages: Make checking for status message consistent with backend.
Adds a check for newline that was present on backend, but missing in the
frontend markdown implementation. Updating messages uses is_me_message flag
received from server instead of its own partial test. Similarly, rendering
previews uses markdown code.

Fixes #6493.
2018-01-23 09:26:41 -05:00
Robert Hönig 4cc8c74aaa frontend: Internally refer to bots by ID.
This is done by using a bot's ID instead of email in
the handler methods for bot_data.bots and bot_data.services,
and updating all code paths involved.
2018-01-23 07:29:00 -05:00
Robert Hönig bd6fa385a5 frontend: Add outgoing webhook config entries to the "edit bot" menu.
This allows users to edit an outgoing webhook's endpoint URL
and interface type after it has been created.

Fixes #7411.
2018-01-23 07:29:00 -05:00
Robert Hönig 9a15c4e3ff Add bot services to page_params.
This is the first step for allowing users
to edit a bot's service entries, name the
outgoing webhook configuration entries. The
chosen data structures allow for a future
with multiple services per bot; right now,
only one service per bot is supported.
2018-01-23 07:29:00 -05:00
Robert Hönig 06fb868482 Send an event to bot owners when a bot is created. 2018-01-23 07:29:00 -05:00
Rishi Gupta 069ab33615 docs: Add gsoc-ideas.md. 2018-01-23 01:27:21 -08:00
Vishnu Ks 036dc53d20 messages: Rename last_visible_message_id to first_visible_message_id. 2018-01-22 19:53:44 -08:00
Greg Price ff29fe07be provision: Install virtualenv on xenial too.
As the commit message on 680381c9d which added this for stretch says,
we need this dependency when on xenial as well.  Add it there.
2018-01-22 19:36:52 -08:00
Vishnu Ks b762b839d4 stripe: Make newly added card default source. 2018-01-22 19:31:17 -08:00
Greg Price 2a59b2d2ac install: Work around a bug in the (our) Debian package for camo.
Before this fix, the installer has an extremely annoying bug where
when run inside a container with `lxc-attach`, when the installer
finishes, the `lxc-attach` just hangs and doesn't respond even to
C-c or C-z.  The only way to get the terminal back is to root around
from some other terminal to find the PID and kill it; then run
something like `stty sane` to fix the messed-up terminal settings
left behind.

After bisecting pieces of the install script to locate which step
was causing the issue, it comes down to the `service camo restart`.
The comment here indicates that we knew about an annoying bug here
years ago, and just swept it under the rug by skipping this step
when in Travis. >_<

The issue can be reproduced by running simply `service camo restart`
under `lxc-attach` instead of the installer; or `service camo start`,
following a `service camo stop`.  If `lxc-attach` is used to get an
interactive shell, these commands appear to work fine; but then when
that shell exits, the same hang appears.  So, when we start camo
we're evidently leaving some kind of mess that entangles the daemon
with our shell.

Looking at the camo initscript where it starts the daemon, there's
not much code, and one flag jumps out as suspicious:

  start-stop-daemon --start --quiet --pidfile $PIDFILE -bm \
    --exec $DAEMON --no-close -c nobody --test > /dev/null 2>&1 \
    || return 1
  start-stop-daemon --start --quiet --pidfile $PIDFILE -bm \
    --no-close -c nobody --exec $DAEMON -- \
    $DAEMON_ARGS >> /var/log/camo/camo.log 2>&1 \
    || return 2

What does `--no-close` do?

 -C, --no-close
     Do not close any file descriptor when forcing the daemon
     into  the  background  (since version 1.16.5).  Used for
     debugging purposes to see  the  process  output,  or  to
     redirect  file  descriptors  to  log the process output.

And in fact, looking in /proc/PID/fd while a hang is happening finds
that fd 0 on the camo daemon process, aka stdin, is connected to our
terminal.

So, stop that by denying the initscript our stdin in the first place.
This fixes the problem.

The Debian maintainer turns out to be "Zulip Debian Packaging Team",
at debian@zulip.com; so this package and its bugs are basically ours.
2018-01-22 18:55:46 -08:00
Greg Price 6e7ae9a239 test-install: Run installer under eatmydata.
This is a tool that throws away `fsync` calls and other requests for
the system to sync files to disk.  It may make the install faster; for
example, if it has to install a number of system packages, `dpkg` is
known to make a lot of `fsync` calls which slow things down
significantly.  Conversely, if there's a power failure in the middle
of running a test install, we really don't mind if the test install's
data becomes corrupt.
2018-01-22 18:55:46 -08:00
Greg Price eb25928674 test-install: Allow the installer to move the install tree aside.
When the install script is successful, one of the final things it
wants to do is to move the tree that Zulip was installed from into the
deployments directory.  It can't do that, at least not in a naive way
with `mv`, if the tree is actually a mount point.  So, stick the tree
inside some other directory that we create just for the purpose of
being the mount point and containing the install tree.
2018-01-22 18:55:46 -08:00
Greg Price 33a0a00705 production test suite: Use installer's own `--snakeoil-cert`. 2018-01-22 18:55:46 -08:00
Greg Price b0a9117e80 test-install: Pre-install a few more dependencies in base image. 2018-01-22 18:55:46 -08:00
Greg Price 0e3ab4d437 test-install: Share the pip cache across installs.
This saves several minutes off the install time.  Sadly pip still
clones Git repos for dependencies that point to them, but for many
others (not all? not sure) it just gets a wheel from the cache.
2018-01-22 18:55:46 -08:00
Greg Price 69ba6ad6d7 test-install: Let installer handle the snakeoil cert. 2018-01-22 18:55:46 -08:00
Greg Price cef8549ec6 install: Add --snakeoil-cert option.
This provides a major simplification for non-production installs,
including our own testing (it's already in both the test-install
harness script and the "production" test suite) as well as potential
admins evaluating Zulip.

Ultimately this should probably be the default behavior, with perhaps
something shown to admins on the web as a reminder and link to help on
installing a better certificate.  For now, pending working through
that, just get the behavior in and leave it opt-in.
2018-01-22 18:55:46 -08:00
Greg Price ac88f8ae1b setup-certbot: Stop automatically "agreeing" to the LE TOS.
It's not appropriate for our script to pass the `--agree-tos` flag
without any evidence of the user actually having any knowledge of,
let alone intent to agree to, any such ToS.  Stop doing that.
Fortunately this script hasn't been part of any release, so it's
likely that no users have gone down this path.
2018-01-22 18:55:46 -08:00
Greg Price 7b47cca67e test-install: Pre-install two more dependencies in base image. 2018-01-22 18:55:46 -08:00
Greg Price 525b136f10 install: Install curl.
The third-party `install-yarn.sh` script uses `curl`, and we invoke it
in `install-node`.  So we need to install it as a dependency.

We've mostly gotten away with this because it's common for `curl` to
already be installed; but it isn't always.
2018-01-22 18:55:46 -08:00
Greg Price 07969a2b0c test-install: Share the tarball directory between host and container.
This greatly simplifies iterating on changes to the installer and
associated code: just edit in the shared directory (or edit in your
worktree and rsync to the directory), and rerun.

With this change, the form with a directory is now really the main
way to run the script; the form accepting a tarball is really just
a convenience feature, unpacking the tarball and then proceeding with
that directory.
2018-01-22 18:55:46 -08:00
Greg Price d7e2190b85 test-install: Pass options through to the installer.
This will facilitate testing interesting installer features
using its own CLI.

On my laptop, with a recent base image (updated a few days ago with
`prepare-base`), it takes just 7 or 8 seconds to get to the installer
running, as timed by passing `--help` so that the installer promptly
exits.
2018-01-22 18:55:45 -08:00
Greg Price de7abd8f78 test-install: Upgrade CLI parsing, with getopt.
This will let us add more options without the CLI collapsing under
its own weight.
2018-01-22 18:55:45 -08:00
Cynthia Lin 7d8cd37035 modals: Dynamically replace keyboard shortcuts for Mac OS.
Fixes #3577.
2018-01-22 19:41:17 -05:00
Cynthia Lin d449fcb309 user docs: Dynamically replace keyboard shortcuts for Mac OS. 2018-01-22 19:41:17 -05:00
YJDave 2691c27126 stream settings: Hide create-stream-btn if user not allowed. 2018-01-22 18:26:36 -05:00
YJDave 01510c1e8f all streams: Show lock-icon for subscriber count, if user can't access.
Previoulsy, we display "0 subscribers" if user can't access stream's
subscribers. Replace subscriber count with lock icon in case of
unsubscribed private stream, in "All stream" list.
2018-01-22 18:26:36 -05:00
YJDave 81599cf906 stream settings: Display warning if user can not access subscribers.
Display warning, saying "You can not access private stream subscribers,
in which you aren't subscribed", if user can not access subscribers;
instead of showing zero subscriber to stream.
2018-01-22 18:26:36 -05:00